COCOHEHI BESDHS 



£D 136 096 



C6 Oil 121 



AOTHOB 
TITLE 

INSTITOTION 
SPOHS A6ENCI 

FDB DATE 
CONTBACT 
NOTE 



EDBS PBICE 
DESCBIPTOBS 



Haider, Leopold 0.; And Others 

Horse Crisis Intervention Center "Project Advance"*. 
Final Beport. — 

Behavior Service Consultants, Inc., Greenbelt, 
District of Columbia Public Schools, fiashington, D.C. 
Dept. of Besearch and Evaluation. 

Jul 76 

0683-AA-HS-O-er-GA 

62p.; Attachments and appendices (pp. 55-111) not 
filmed due to marginal legiblity. 

11F^$0.83 HC^$3*50 Plus Postage. 

4(0emonstration Projects; ^^Educational Alternatives; 
4t£valuation Methods; winter vention ; Interviews; 
^Program Evaluation; Psychological s.eryices; 
Questionnaires; Secondary Education 



ABSTBACT 

An evaluation of the ''Project Advance" program at the 
Horse Crisis Intervention Center vas conducted. Beviejf of documents, 
formal and informal intervievs, questionnaires and direct 
observations vere the methods used to assess the correspondence 
between the objectives of the program and its achievements. The 
evidence, based on findings from analyses of all the data, points to 
a program which shows a high degree of correspondence betveen its 
objectives and its achievements. Baw data, findings based on analyses 
of these data, and conclusions are included in the Final Evaluatipn 
fieport. Becommendation to continue the program, with a few 
modifications of certain aspects of the prograsi, are provided in the 
FdLnal Evaluation Beport. A discussion of tiie roles of the inside and 
outside e valuators and a presentation of suggested ways to improve 
the evaluation of this and other programs are also included in this 
report. (Author) 



* Documents acquired by EBIC include many informal unpublished ^ 
4t materials not available from other sources. EBIC makes every effort * 
^ to obtain the best copy available. Nevertheless, items of marginal * 

* reproducibility are often encountered and this affects the quality * 
^ of the microfiCihe and hardcopy reproductions EBIC makes available 

* via the EBIC Document Beproduction Service ^EDBS) . EDBS is not 
4< responsible for the quality of the original document. Beproductions^ 

* supplied by EDBS are the best that can be made from the original. * 




U.S. DEPARTMENT OF HEALTH. 
EDUCATION ft WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION 

THIS DOCUMENT HAS BEEN RgPRO 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATrON ORIGIN 
ATINGIT POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY RgPRg 
SENT OFFICIAL NATIONAL INSTITUTE DF 
EDUCATION POSITION OR POLICY 



Page i^ SS^l/l ^as / were 

Mkjflrng/ Removed From. This Docunfient 

Prior To Bemg Submltfed To EDRS 



BEHAVIOR SERVICE CONSULT ANTS* 



133 CENTERWAY BOX 97. ^ 
GREENBELT, MARYLAND 20770 • USA 
TEL. (301) 474-2146 
CABLE: BEHAVIOR. WASH. D.C; 




FINAL REPORT 



Public Schools of the District of Columbia 



Morse Crisis Intervention Center 
"Project Advance" 



Office of Planning, Research and Evaluation 
Division of Research and Evaluation 



July 1976 



Morse Crisis Intervention Center - Project Advance 
(R.F.P. No. P.S. 7607 (Part 1) 
N.S. Contract No. 0683-AA-NS*0-6-GA 



Coordinated under the Direction of: 

Division of Research and Evaluation 
Room 1013 
415 - 12th Street, N.W. 
Washington, D.C. 20004 



Prepared by: 

Leopold 0. Walder, Principal Investigator 
Linda J, Citnarusti 
Marcel la G. Walder 

Behavior Service Consultants, Inc. 
133 Centerway 
Greenbelt, Maryland 20770 
(301) 474-2146 



3 



Table of Contents 

Page 

I • Abstract 1 

II. Ptu^ose 1 

III. Background 1 

IV. Evaluation Design 2 

V. Evaluation Methods 7 

A. Face-to-Face Interview 7 

B. Paper and Pencil Questionnaire 9 
C ; Direct Obsei*vation 11 

D. Test - Retest Agreement of the Measuring Instmments 13 

E. Validity of the Measuring Instruments 14 

F. In--depth Observation of the Progress of a Subgroup 14 
of Students 

VI. Results 14 

A. Introductory Remarks 14 

B. The Data Gathered 15 

1. Face-to-Face Interview 15 

2. Paper and Pencil Questionnaire 15 

3. Direct Observation 16 

C. Results by Hypothesis 16 
VII. Conclusions 44 

A. By Hypothesis 44 

B. Discussion of Role of Evaluators and Ways to Improve 50 
Evaluation of Program 

VIII. Recommendations 51 

IX. Stommary 52 



4 



i 



Page 



X. Attachments 

A. Face-to-Face Interview/Student 55 

B. Face-to-Face Interview/Teacher 57 
Paper and Pencil Questionnaire/Student 61 

D. Paper and Pencil Questionnaire/Teacher 70 

XI, Appendices 77 

A. Program Schedules for Four Morse Center Students 78 

B. Typical Month's Schedule 78,5 

C. Guided Group Interaction Log 79 

D. Schedule of Field Trips and Activities 80 
Morse Home and School Interaction Team Schedule 82 

F. 1973-1974 Follow-up Report Form 83 

G, Morse Center Staff Development Workshops 84 

H, Letter of Contact sent to Receiving School 84 

I. List of Topics to be Discussed at June 9^ 1976 Staff ftS 

Meeting 

J. Schedule of Site Visits by Evaluators to Morse Crisis 86 
Intervention Center 



5 



ii 



Table of Tables 



Title 



Number and Percentage of Students 
Rating Individualized Instructional 
Program 

Number and Percentage of School 
Personnel Rating Individualized 
Instructional Program 

Number and Percentage of School 
Personnel Interviewees Rating 
Individualized Instructional 
Program 

Evalua tors' Observations of 
Classroom Activities 

Number and Percentage of Students 
Rating Ongoing Assessment of 
Students' Progress 

Number and Percentage of School 
Personnel Rating the Psychotherapeutic 
Counseling Program 

Number and Percentage of Students 
Rating Aspects of Guided Group 
Interaction Sessions 

Number and Percentage of Students 
Expressing Positive Feelings Toward 
School 

Number and Percentage of Students 
Rating Enrichment Program 

Number and Percentage of School 
Personnel Rating E?>richment Program 

Number and Percentage of School 
Personnel Rating Activities Relevant 
to Preparing Students to Return to the 
Regular Public School System 

6 



Table of Tables (Contd.) 



Title 



Number and Percentage of School 
Personnel Rating Pre and Inservice 
Training 

Number and Percentage of School 
Personnel Rating Ongoing Assessment 
of Student Progress 

Number and Percentage of School 
Personnel Rating Ongoing Assessment 
of the Functioning of the Program 

Number of Endorsements by Students 

of Aspects of the Enrichment Con^onent 

of the Morse Center Program 

Number and Percentage of Students 
from Main Administration Rating 
Aspects of Program 

Number and Percentage of Students 
from Pretest Administration Rating 
Aspects of the Program 



iv 



We would like to acknowledge those persons whose assistance 
made possible the implementation and evaluation of this project. 



The Public Schools of the District of Columbia 

Vincent E, Reed, Superintendent of Schools 

Harris M. Taylor, Associate Superintendent for State 

Admini strati on 

Mildred P. Cooper, Assistant Superintendent for Division 

of Research and Evaluation 

Lavolia W. Vails, Coordinator for Title III Project 

Evaluations 

Elsa Miller, Title III Advisory Council Chairperson 
Harris M. Taylor, Director of Federal Programs 
Grace H. Davis, Coordinator for ESEA, Title III Projects 
Helen W. Holmes, Project Director 



8 



V 



I. Abstract 



An evaluation of the "Project Advance*' program at the Morse Crisis 
Intervention Center was conducted. Review of documents ^ formal and infor- 
_jnal<.interviews, questionnaires, and direct observations were the. methods 
used to assess the correspondence between the objectives of the program 
and its achievements. The evidence, based on findings from analyses of 
all the data, points to a program which shows a high degree of correspon- 
dence between its objectives and its achievements. 

Raw data, findings based on analyses of these data, and conclusions 
are included in the Final Evaluation Report. Recommendations to continue 
the program, with a few modifications of certain aspects of the program, 
are provided in the Final Evaluation Report. A discussion of the roles 
of the inside and outside evaluators and a presentation of suggested ways 
to improve the evaluation of this and other programs are also included 
in this report. 

II. Purpose 

The purpose of this report tc provide information concerning the 
findings of the evaluation of "the ''Project Advance" program at the Morse 
Crisis Intervention Center. A major issue is the amount of correspondence 
between the objectives of the program and its accomplishments. A second 
important issue is determination of the events or circumstances that have 
influenced the achievement of program goals. A third issue is examination 
of the techniques used by project personnel to perform the internal eval- 
uation of the program. A fourth issue is the investigation of ytays in 
which the internal and external evaluators can collaborate to improve the 
evaluation of this program. A fifth issue is the presentation of suggested 
ways to improve the evaluation of programs in general. 



III. Background 

A critical problem that faces all school systems is the development 
of alternative educational programs for those students who are unable to 
function in the regtilar classroom. Students with both emotional and 
behavior problems all too often have academic problems as well. The task 
that faces educators is, therefore, a complex one. These students must 
be provided with: (1) assistance in personal and social adjtistment, (2) 
remedial work in many academic areas, especially reading and mathematics , 
and (3) a learning environment that is capable of sparking their interest 
sufficiently to compensate for the apathy and negative feelings toward 
school that have developed through repeated failxxre to perform success- 
fully. 

The available resources designed to work with such students (e.g. ^ special 
education classes, resource teachers for remedial work, school-based 
coxmselors) all too often are not able to provide the kind of long-term 
intensive experience that is necessary in order to intervene successfully 
in the patterns of failure that have been established and reinforced. 



Clearly a comprehensive program that is designed to meet the special 
and critical needs of these students is of utmost importance • 

Morse School, a D«C« p\;blic school facility located on R Street in 
the Northwest section of Washington, D«C«| has had a four year history 
of providing rehabilitative care to Black, inner city students identified 
as unable to function within the regular school program. 

During the course of their work with these youths, the staff of the 
Morse School realized that additional and/or more intensive services would 
have to be provided in order to meet the needs of the students currently 
enrolled and to work with a greater number of students at any one time. 

A strong psychotherapeutic program would be necessary in order to 
assist the students with pez^onal growth and emotional adjustment. The 
majority of the students would need remedial work in both reading and 
math. The wide range of abilities in different academic areas, coupled 
with great diversity in reading level, would necessitate a highly individ- 
ualized curricxilum. Since years of failure had reinforced an apathetic 
attitude toward school and learning, a different approach to learning 
would be necessary in order to arouse the student's interest. Sorely 
needed opportunities for participation in sports activities and e:q)osure 
to cultural exhibitions and events, as well as career development educa- 
tion, would also be highly desirable components of a comprehensive educa- 
tional program. 

A proposal for the development and implementation of such a program 
was written by the Assistant Principal of the Morse School. The objec- 
tives of the program would be to provide 60 junior high school students 
who were referred to the Morse Crisis Intervention Center because of 
disruptive behavior in the regular school with (a) psychotherapeutic 
services, (b) an instructional program that emphasizes individualization 
through the modified Open Classroom technique, and (c) a program of 
enrichment that will reinforce the educational program. The proposal 
was submitted to the D.C. Public Schools, funding was granted and 
•'Project Advance*' was operationalized in 1973. 

IV. Evaluation Design 

The evaluation design included the development of hypotheses to be 
tested, the selection of the variables to be measured, determination of 
the quality of measurement, identification of the sources of relevant 
data, processing of these data to obtain findings, and presentation of 
the findings, conclusions, and recommendations relevant to the evalua- 
tion of the program. 

The bases for the development of the hypotheses to be examined and 
the selection of the variables to be measured came from several sources. 
One source was the description of the program provided to the evaluatcrs 
by the personnel of the D.C. Public Schools. Another important soiarce 
was discussions with the participants (students, teachers, and adminis- 
trators) of the project. Other sources of hypotheses were suggestions 
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available to the evaltiators from their readings about and discussions 
concerning development of individualized curricula and their knowledge 
of psychotherapeutic techniques. 

These hypotheses were used to assess the correspondence between the 
objectives of the program and its achievements. Each of the hypotheses 
was examined by studying corresponding variables which are assessed by 
the use of questions designed to focus on a specific hypothesis, the 
hypotheses, with paraphrases from the description of the program given 
to txs by the Division of Planning, Research and Evaluation and with 
examples of questions used in the face to face inteirview, the paper and 
pencil questionnaire, and in some cases other procedures such as direct 
observation techniques, are presented below: 

Hy pothesis I (as stated in the Program Description) 

Sixty junior high school students will participate in an 
instructional program that emphasizes xndivxdualjLzation through 
the modified Open Classroom technique. 

In order to achieve this objective, the project personnel 
will observe the students in a variety of settings in order to 
determine their interests and abilities. An individualized 
instmactional program built around his/her interests and abili- 
ties will be developed for each student. 

Classes in Mathematics, Language Arts, Social Studies, 
Health Education, Shop, and Arts and Crafts will be held on 
Monday, Tuesday, Thursday, and Friday. These classes will be 
conducted using the Open Classroom technique and will allow 
students to take an active part in their own learning. 
Schedules for each student will be reviewed periodically with 
the student to insure that his/her needs are being met. 

The following questions, taken from face to face intex^riews (Attach- 
ments A and B) and paper and pencil questionnaires (Attachments C and D), 
were among those used to determine whether such a program was implemented 
at Morse School. 

Question 9 (Teacher paper and pencil questionnaire): An individualized 
instructional program has been developed for each student # 

Question 14 (Student paper and pencil questionnaire): My teachers 
like it if I ask them questions dxiring class. 

Question 9 (Teacher face to face interview): How, and with what fre- 
quency, do you evaluate the progress of students in your class? 

Hypothesis II (as stated in the Program Description) 

Sixty junior high school students will be provided with a 
psychotherapeutic counseling program that will focus on reducing 
and/or eliminating disturbing behaviofT 
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In order to achieve this objective, a strong intensive 
program of individual and group counseling and therapy vdll 
be conducted by a qualified psychiatric social worker* This 
person will work closely with both the student and his/her 
parent in an effort to open avenues of understanding of pro- 
blems in behavior and to provide assistance in coping with 
such problems. He/she will also provide consultative help 
to the staff. 

The following questions, taken from face to face interviews (Attach- 
ment A and B) and paper and pencil questionnaires (Attachments C and D) , 
were among those used to measure whether the students have been provided 
with the opportunity to participate in a psychotherapeutic counseling 
program that focuses on reducing and/or eliminating disturbing behavior. 

Question 18 (Teacher face to face interview): How well do the 
psychotherapeutic consultants meet the needs of the students and the 
objectives of the program? 

Question 25 (Student paper and pencil questionnaire): I feel okay 
about talking about what's on my mind during the Group Sessions. 

Hypothesis III (as stated in the Program Description) 

Sixty junior high school students will participate in a 
program of enrichment activities that include sports activities; 
opportunities ^or career development; and art, music > and drama 
activities > * 

In order to achieve this objective, every Wednesday will 
be designated as "Special Projects Day." On these days, and 
at other times during the week, students will participate in 
sports activities such as bowling and horseback riding; go on 
field trips to museums, theaters, and other places of educa- 
tional interest in the iretropolitan Washington, D.C. area; 
and participate in special programs such as bake sales and 
seminars, etc. An effort will be made to include the parents 
of students in these activities. The activities will be 
designed to provide the student with opporttinities to release 
tensions, expose him to cultural activities, offer him an 
opportunity to familiarize himself with various careers, and 
to gain an awareness of available job opportunities. 

The following questions, taken from the student paper and pencil 
questionixaire (Attachment C), were used to elicit information from the 
students about the enrichment component of the program: 

Question 19 (Student paper and pencil questionnaire): The places 
we go to on our field trips are places where I can see and meet people 
who work at different kinds of jobs. 

Question 20 (Student paper and pencil questionnaire): The field 
trips give me a chance to do some sports activities. 
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Hypothesis IV (as stated in the Program Description) 



. All : pTO vrill focus on the objective o f 

v^.^:^^ participants to return to the regular : 

-' public'-':s6hoolMsystem>v ^' ■/ • ' 'Cy • ' : - '■■■'{[■•''.j-'W:--: " \ 

^ I^^ achieve this objective / the program will be 

designed to provide st^ and support which 

>dll prepare them to function again in the regular public 
school program* There^^ w^ be sr 4ures for tran- 

sition of a student back into ' ogram: 

a) Student counseling wiix off^^ed to the student. 

b) A school*-based conference (including parents, 
counselor , and admini strait or from the sending 
school ) will be held at the receiving school . 
Procedures for programming for a^^ 
embtiohal support will be diacussed. 

c) Contact with departing students will be main- 
tained by Morse Crisis Center personnel for a 
period of one year after the student returns 
to a regular program or leaves Morse School. 

The following questions, talten from the paper and pencil questionnaire 
(Attachments C and D), were among the measures used to determine vrtiether 
the focus of program activities was on prejparing students to retiun to 
th<5 regular public school program; 

Question 24 (School personnel paper and pencil questionnaire) - The 
criteria by which a student is identified as ready for transition back to 
a regular school program are clear and well defined. 

Question .^5 (School personnel paper and pencil questionnaire) 
Students who are placed back into a regular school program receive follow- 
up service for at least one 'year. ■ ■ ^■■■.■■^■■'v^^ 

Question 34 ( Student paper and pencil questionnaire) r^^^A leave 
here, 1^^^^^^^ touch with me. 

Hypothesis V (as stated in the Program Description) 

All members of the staff will participate in pre- and 
in-service training seminars that focus on eiftective teaching 
techniques # ■ ■ ■ / . , ■ ,„/. '''■-■■^C-''^'. vV\': ' ' 

In order to achieve this objective, the staff at Morse 
Crisis Interviention Center will receive pre-service and in- 
service li*aining from D.C. Teachers College and 3pecial^^^ 
/Education Personnel in the specialized techniques of :^ manag- 
ing disruptive behavior in the classroom, teaching in a 
modified Open Classroom setting, individualizing instruction 



to meet the special needs of non-achieving and alienated 
children^ and the use of guided group interaction as a 
counseling tool. In addition^ staff development seminars^ 
which vrill focus on issues related to teaching in the 
Morse Crisis Intei*vention Center, will be held on a 
regular basis. 

The following questions > taken from face to face interviews, (Attach- 
ments A and B) and the paper and pencil questionnaires (Attachments C and 
D), are examples of those used to elicit information from project personnel 
concerning their pre- and in-service training: 

Question 16 (School personnel paper aiii pe>^c . iestionnaire) ; The 
issues dealt with during the Wednesday in-service training seminars are 
relevant to the objectives of 'Troject Advance** 

Question ^ 16 (School personnel face to face interview); What pre- 
service training opportunities were available to you? 

Hypothesis VI (as stated in the Program Description) 

There will be ongoi rig ssessment by project personnel 
of stu5ents ' progress o .'^he functioning of the program 
as a whole. 

In order to achieve the objective of ongoing assess- 
ment of students' progress, a fonaal pretest will be admini'?- 
stered to each student at the beginning of the school year 
(or his/her date of arrival at the Morse Crisis Intervention 
Center). Other informal assessment techniques, such as 
progress charts for academic and social skills, student 
interviews, teacher observations, psychological reports, 
etc. y will also be utilized. Each student's progress will 
be assessed semi-annually by project personnel in order to 
determine readiness for return to the regular D.C. Public 
Schools program. 

The functioning of the program as a whole will be 
evaluated by the staff and -liie students through group 
discussions, questionnaires, and written narrative reports. 
In addition, professional services will be contracted for 
the external evaluation of the project. The outside eval- 
uation consultant will be under the direction and super- 
vision of the Division of Research and Evaluation. 

The following questions, taken from the face to face interviews 
(Attachments A and B) and the paper and pencil questionnaires (Attach- 
ments C and D), were among those used to gather information about the 
internal evaluation of student progress and the program in general: 

Question 9 (School personnel face to face interview): How, and with 
what frequency, do you evaluate the progress of a student in your class? 

-6- 



14 



Question 12 (Student paper and pencil questionnaire): l/ly work is 
checked by my teacher every day. 

Question 22 (School personnel p^^per and pencil questionnaire): There 
is ongoing assessment of each student's social and personal adju;itment. 

V. Evaluation Methods 

The data sources identified were: (1) the program description pro- 
vided to us by the Division of Planning^ Research and Evaluation^ (2) other 
documents and records related to project activities^ (3) student and staff 
participants of the project, and (4) the observation of ongoing project 
activities • 

The measuring instrumen t were used to gather data were (1) face 

to face interviews, (2) pi er ea ^^^iicil questionnaires, and (3) direct 
observations. Each measuri ^'^ ument was tised to obtain information 
about, more than one hypothesis . Our search for information was; organized 
around the hypotheses. Therefore, we identified variables that were 
relevant to each hypothesis and addressed ourselves to these variables 
in the development of our data-gathering instruments. The face to face 
interview, which served as a basis for the development of the paper and 
pencil questionnaires, allowed us to obtain a richness of detail about 
a participant's perception of a project that is difficult to achieve by 
a structiired questionnaire format. 

The paper and pencil questionnaires allowed us to obtain a large 
quantity of information from a large number of persons and to standardize 
the questions used to elicit such information. 

Direct observations yielded a rich return of information about on- 
going activities, participai - interactions;, and project documents and rec- 
ords. They enabled us to assess the correspondence between what we heard 
about the project from program participants and what we saw . The fact 
that the presence of outside observers is likely to have some effect on 
the manner in which the participants carry out the activity that is being 
observed should be taken into account when examining data obtained from 
direct observations. 

The evaluation suffered from a general problem in the design of this 
project. The evaluation component, both inside and outside of the project, 
did not provide for necessary comparison ^d control groups and adequate 
before and after measures. This deficit is common to many projects similar 
to this one. A discussion of this general problem will be included in 
the conclusions section, 

A, Face to Face Interview 

1. Purpose 

The face to face interview allowed us to obtain a large amount 
of inforaation about various aspects of the program and about the 



program in general from a limited number of participants. Although 
the time expenditure on the part of the particip^ts and the eval- 
xiators was greater than with other types of measuring instruments, 
the richness of detail and the variety of issues addressed provided 
us with information that would have been difficult to obtain by 
measuring instruments with more structured response formats , such 
as paper and pencil questionnaires • In addition to providing; us 
with invaluable information, the responses obtained from the face 
to face interviews served as a base for the development of the 
paper and pencil questionnaire. 

2. Development 

The project personnel face to face interview was developed on 
the basis of (a) information about program objectives and 2u:t^ 
contained i> 4$ cription of the program klven to us by the D.C. 
School' v^f ' ^^ijining, Research and Ev|Bimtion, (b) the 

hypbthesciu m be e^tiuidned,. (c)^ o 

activities i and participants * interactions , ( d) informal intidr^ievns 
with program participants conducted during the initial stages of 
the current evaluation, and (e) earlier versions of the interview 
which were used to develop the final version of the face to face 
interview. 

The student face to face intei*view was developed on the basis 
of the sources listed above. Two informal interviews with students 
were conducted prior to interviewing students formally. Occasion- 
ally, a word or two was exchanged with several students in the 
halls or classrooms, but these contacts will not, for the purposes 
of this evaluation, be regarded as informal interviews. 

3. Description 

The questions included in both the project personnel face to 
face interview (included as Attachment A in the Attachment Section 
of this report) and the student face to face interview (included 
as Attachment B in the Attachment Section of this report) are open- 
ended in structure. The questions and the questionnaire formats 
were designed to maximize the freedom of participants to respond 
in whatever manner they chose and to give as much or as little 
information as they chose. The questions contained in both inter- 
views deal with various components of the program, such as the 
psychotherapeutic services and the individualized instructional 
program. The student interview contains questions specifically 
designed to elicit information about observations of and attitude 
toward the program and project personnel. 

4. Procedure 

All participants were interviewed individually. All inter- 
views were conducted by one interviewer. The setting of the 
interviews varied, with interviews conducted in the Project 



Administrator's office, the Multi-Purpose room, and the offices 
of project nersonnel. The interview time for students ranged 
from five CO ten minutes* Interview time for project personnel 
ranged from 15 minutes to one and one-half hours. 

5. Scoring 

The responses received to the questions were categorized 
according to relevance to the empirical hypotheses, and were 
analyzed accordingly. 

B. Paper and Pencil Questionnaire 

1. Purpose 

The paper and pencil questionnaires were developed in order 
to make manageable and standardized the gathering of relevant 
information from a l^ge number of participants. They also 
provided us with an opportunity to obtain a measure of the relia-- 
bility of a data-gathering instrument. By administering the 
questionnaire twice to selected subgroups of participants, we 
were able to obtain two sets of responses to the same questions. 
These responses were then examined for similarities and differ- 
ences. 

2. Development 

The student paper and pencil questionnaire was developed 
based on several sources: (a) a description of the program and 
its objectives given to us by the D.C. School's Division of 
Planning, Research and Evaluation, (b) the hypotheses to be 
examined, (c) information obtained during face to face interviews 
with participants, (d) obsejrvations by the evaluators of ongoing 
program activities, and (e) an earlier version of the student 
questionnaire that was pretested during the current evalitation. 
This initial version of the student questionnaire, containing 
both untested questions and questions which had already m 
tested in face to face interviews^ was administered to ; a' s 
of four students. Feedback from the respondents ena-bled us to 
revise and refine the questionnaire. Specific questions were 
reworded, added, or deleted, and the format of the section 
designed to elicit demographic information from the respondent 
was restructured. The instructions for responding to the question- 
naire were reworded in order to make them more precise. 

The project personnel questionnaire was developed based on 
the sources listed above. 

3. Description 

The student paper and pencil questionnaire (included as 
Attachment C in the Attachment Section of this report) is composed 
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of questions with a Likert scale response format. The respondent 
is asked to choose among five alternative responses that range 
from an "All of the time" to.a "None of the time" response. The 
respondent indicates his/her choice by circling the number that 
represents the frequency of occurrence of a specific item. The 
questions deal with aspects of the program such as Guided Group 
Intejraction sessions^ freedom to move about during a class, field 
trips, etc. The respondent is asked not only about Aether 
specific program activities occur but also whether or not she/he 
enjoys and learns from them. 

The project personnel questionnaire (included as Attachment 
D in the Attachment Section of this report) contains questions 
which deal with various aspects of the Project Advance program 
as well as participants' views about specific activities to 
the program in general , the response ^^^^^f^^^ . _ 

that of the student qu^ / ^ 

4. Procedure 

The unrevised student questionnaire was administered by an 
evaluator to a group of four students in the G^ 
Office at Morse Center. The time required to explains t^^ 
and response format of the questionnaire to the students and hav 
them complete the questioxmaire rianged from 14 to 2i2 minutes, with 
the differences resulting from variations in readings comprehension, 
and response speeds of the students. In order to facilitate the 
administration of the questionnaire to one student, whose reading 
level did not allow him to read and comprehend many of the words 
included in the questionnaire, the evaluator read aloud to this 
student each question and its response alternatives. After the 
questionnaires were completed, questions designed to elicit 
evaluative feedback from respondents and the guidance counselor 
about specific questions and phrases as well as the response 
format itself were posed by the evaluator. The guidance counselor 
was extremely helpful in offering suggestions for ways to revise 
the questionnaire so that it could be read, comprehended, and 
responded to by the majority of students. 

The final version of the questionnaire was administered by 
the evaluators to the student population. A tape recording of the 
questionnaire was made and played in each classroom while the 
students followed along on their questionnaire forms and circled 
the response they chose. This was designed to give some additional 
help to slow readers who would have been unable to answer the 
questions independently. 

The final version of the project personnel questionnaire was 
administered to the teachers by an evaluator during a Wednesday 
afternoon staff development meeting in late May. The time required 
to complete the questionnaire was approximately 10 to IS.iminutes, 
depending on response speed of the participant. 



5. Scoring 

The response formats of both paper and pencil questionnaires 
were designed in such a way as to facilitate computer processing. 
Each was categorized according to relevance to the empirical 
hypotheses and was analyzed accordingly. 

Direct Observation 

1. Ptu-pose 

Direct obsei^vations enabled^^the evaluators to gain first hand knowledg 
of many of the aspects of the program listed in the description of 
the program and discussed d\iring interviews with program partici- 
pants. 

2. Development 

Prior to and during all site visits we tried to acquaint ou3>- i 
selves as much as possible with the various aspects of the program. 
Our readings about the progrsun and initial discussions with program 
participants, as well as our initial observations of the physical ' 
facilities, on^;oing ^^tivities and participants' interactions, 
enabled us to devel 6 a checklist of activities and materials 
relevant to our evaluation. 

3. Description 

The following aspects of the program were among those considered : : 
relevant to our direct observations: > 

a. Staff Development v 

(1) documents and materials related to courses offered Jvi 
as pre-sei^ice training 

(2) in-service training seminars and courses 
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b. Physical Facility 

(1) flexible use of rooms arid space 

(2) kinds and location of furniture 

(3) kinds and use of equipment and educational materials 

c. Educational Component Si 

(1) academic class schedules - ll 

(2) classroom activities (e.g. Open Classroom aspects) g 

(3) sti«4ent-teacher interactions 

(4) slsiirient programs, foMers, lesson pMns 

(5) teacher-student raidio 

d. Psychotherapeutic Component 

(1) Guided Group Interaction sessions 

(2) schedules for individual and group therapy 

(3) interactions between consultants and: staff 

J~~ ^ ^ 19 



(4) procedures used to select students for individual 
and/or group therapy 

e. Enrichment Component 

(1) schedule of field trips 

(2) participant interactions and student behavior on 
field trips 

(3) special projects and activities 

(4) parent involvement 

(5) communication methods (e.g. newsletters ^ bulletins ^ 
etc.) 

f . Internal Evaluation 

(1) formal pre and posttests 

(a) appropriateness, reliability and validity of 
tests 5ielected 

(b) schedule of administration dates 

(c) test scoies, including summary statistics of 
the scores 

(2) informal evaluation methods 

(a) progress charts 

(b) follow up records 

(c) staff questionnaires 

(d) staff conferences and discussion groups 

(3) semi-annual assessment techniques and records 
(a) procedures used to determine students' 

readiness to re-enter a regular school program 

4. Procedure 

The evaluators,. -cither individually or in pairs, visited class- 
rooms, attended a guided group interaction demonstration, and 
examined records and other documents in order to make observations 
of interactions, activities, and materials relevant to the eval- 
uation of Project Advance. The evaluators used discretion in their 
observation activities so that their presence disrupted what was 
going on as little as possible. Notes were taken at the time of 
the observation. 

Several methods of observation were used. General observations 
are made of all activities and materials occurring during a specific 
time period. Time, interval, and event sampling techniques were 
akso among the observation methods in use during the current eval- 
uation. 

3^ Scoring 

Observaldons were categor: zed according to their relevance to 
jme or more aempirical a priori hypotheses. Consideration was given 
^±D the correspondence of observed events to proposed program 
^activities as. described in the description of the program. 
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D* Test - Retest Agreement of the Measuring Instnuaents 

!♦ Paper and Pencil Questionnaire 

The final version of the student paper and pencil questionnaire 
was administered tvdce to a subgroup of students. The number of 
persons in this subgroup was small since we wanted t€> ^void over- 
btirdening too many people. The two administration- ^^tt« separ^Sid 
by a time interval of approximately ? weeks. The d^t« ea 
administration was chosen to om%e gs little inoonveniaiice as 
possible to the participants, fh^ evaluators did, however, take 
into consideration the fact that it is preferable not to have 
the retest given after a long time lapse. This is important 
because differences in responses as a result of changes in program 
emphases or achievements would be more probable if there was a 
time lapse of several months between the two administrations. 

Since all questionnaires were filled out anonymously, it was naU 
possible to match questionnaire to questionnaire. A comparison 
of the responses of the pretest group (Group 1) to specific 
questions with the responses obtained from Group 2 (main group) 
during the main administration mil be presented. 

Tables XVI and XVII present a coo^arison of first adminis tuition and 
retest administration responses by students to selected questions 
from the paper and pencil questionnaire. Questions were selected 
in the following way: Questions were grouped according to 
relevance to a particular component of the program. At least 
one question from each group was then randomly selected for 
inclusion in the test - retest data analyses. 

Examination of the tablets showed that there is fair to good 
agreement on the majority of the items. A portion of the non- 
agreement of responses may be due to the range of response 
alternatives available, that is, for each question a respondent 
was asked to choose among five alternatives. Some discrepancy 
might be expected with such a range. A second factor which made 
it difficult to compare response percentages fx!om one administration 
to the next was the fact that the pretest grouprwas composed of 
only three students. One response, therefore, carried much more 
weight and could easily skew the group percentage for a specific 
item. With these limitations in mind, a comparison of the percen«- 
tages of "All of the Time" responses for the two groups showed 
that the agreement level is fairly high. 

Initial observations of facilities and participant inter-' 
actions were made by two or more observers at the same time and 
place in order to check for interjudge agreement in the observations. 
Discussion of the observations made revealed a sufMciently high 
level ^ agreement concerning specific interactions^^ behaviors, 
equipment, and materials to allow the observers tosmake further 
observations individually. 
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E. Validity of the Measuring Instruments 



The validity of the measuring instrtunents ^ed in the evaluation 
of the Project Advanc wi^S determined b; malysis of the 

relations among resi^v veit^i by the same grc respo^lltlenit to 

different measuring jlu Hm« lnterrelation#hi|)s among the parti- 

cipants' responses elicited by questions from the face to face inter- 
view and paper and pencil questionnaires, and the direct observation 
data as reported by the evaluators, supported the hypothesis that the 
various data gathering instruments were measuring the same variables. 
The relationship of these reports to the stated objectives of the 
project proposal supports the validities or meanings of the measuring 
devices used in this evaluation. 

F. In-depth Observation of the Progress of a Subgroup of Students 

The eval\iators selected four students whose progress was "followed'V 
through interviews, classroom observations, exananation of test scores 
and student folders. The data obtained from this in-depth study 6^ 
these four students was used to supplement the data obtained from^^ 
data sources and clarify our knowledge of programi activities. 

The four students were chosen, with assistance from the Project 
Director, on the basis of length of time in the program. In order 
to maintain confidentiality, the students will be designated as Boy A, 
Boy B, Boy C, and Girl A whenever referenced for the purpose of 
discussion in this Report. 

The length of time that these four students were at Morse Center 
ranged from almost two school years to one month. At the end of the 
1975-76 school year. Boy A and Girl A will have been at Morse Center 
two months. Boy B for one month, and 3oy C for almost two school years. 

Results 

A. Introductory Remarks 

Some general introductory remarks about the findingsobtained 
from each data gathering procedure shall be made and then findings 
will be discussed in terms of their relevance to the empirical 
hypotheses. 

The findings to be presented in the Results Section: to follow 
showed a program that is well developed and has achievjedia high degree 
of correspondenc© betweenjxts objectives and its accon^piishments. 
The majority of the programi components described in the original 
proposal were implemented and have progressed nicely. Additional 
support services that focus on; (1) helping the teachers develop 
an educational technology based on the Open Classroom approach and 
(2) modifying student-teacher interactions and student program 
development to include increased emphasis on the use of behavioral 
techniques would add to the effectiveness of the program. 



The program in general, however, is basically sound and functions 
a good intervention measure for the population it serves. 

The Data Gathered 

1. Face to Face Interview 

Six formal and numei^ous informal face to face interviews 
with school personnel were conducted by the evaluators. Formal 
telephone interviews with the parents of four students who attended 
Morse School at the time of the evaluation or during the three 
year life of the project were conducted by the evaluatprs • Four 
formal interviews were conducted with the four students whose 
progress was being followed and several students were interviewed 
informally as the evaluators interacted with them in hall WAy« oJ^ 
classrooms during direct observation activities. Interviews were conducted 
with persons from two neighborhood business organizations who » 
because of proximity to the Morse Center, were in a position to 
have first hand knowledge of the Morse Center and its activities. 

All of the participants appeared quite willing to speak 
frankly about their participation in and feelings about Project 
Advance, The school personnel were particularly helpful and, 
indeed, anxious to share with the evaluators their knowledge of 
and attitude toward the project. 

2. Paper and Pencil Questionnaire 

a. Student paper and pencil questionnaire 

The unrevised student paper and pencil questionnaire 
was pretested with a group of four students. Based on their 
responses and evalxaative feedback about the questionnaire 
format and specific questions » as well as consultation with 
the Guidance Counselor, the questionnaire was revised slightly. 

Thirty-six students responded to the revised paper and 
pencil questionnaire. Thirty- three of the respondents were 
male, and three were female. The age of the sinidents^^^ r 
from 13 to 18 years old. Five and one-half percent 
students who responded are 13 years of age. Onef student, or 
2.7 percent of those who responded, is 18. The majority of 
the students (81^) are between 14 and 16 years old L 

Twenty-eight of the 36 students. ( 80 56- of the respon- 

dents) came to the Morse Crisis Intervention Center from "Uie 
District of Coliambia Public Schools System. Three of the 
respondents came from private special education schools in 
the Metropolitan Washington, D.C. area. Three students came 
from private special education schools located outside of 
the Metropolitan Washington, D.C. area. Nineteen students, 
when asked what grade they are now in, said that they were in 
the eighth grade . Five and nine students believed that they 



were in the seventh and ninth grades respectively. Since 
Morse Center opei^ates on an vingraded basis, it is difficult 
to know whether students are basing their grade level on the 
grade in which they were in prior to coming to Morse or where 
they feel that they should be because of chronological age or 
length of time in school. ; 

b. Project personnel questionnaire 

The project personnel questionnaire was administered to 
19 of the 22 regular full-time school personnel .Nine Class- 
room Teachers, four Teache3>-Aides, the Gtiidance Counselor, 
the Social Worker, two Resource Teac two Administra- 

tive Aides responded to the questionnaire. 

Analyses of the data from the responses to the student 
questionnaire and the project personnel questionnaire are 
presented in Tables I through XVI. The data are grouped 
according to relevance to specific program components and/or 
program objectives . Each table contains the data ^rom^a^ 
of responses given to all questions dealing with one of the 
program components previously identified as relevant to this 
evaluation. - 

3. Direct Observation 

Thirteen site visits were made to the Morse Crisis Inte3?vention 
Center. During these visits, observations were made by means of 
standard time, interval, and event sampling techniques; Among the 
behaviors observed were student-teacher interactions, student- 
student interactions, class size, classroom activities, guided 
group interaction ac ti vi ti es , interactions among staf f , and the 
physical faciliiy. Test scores from standardized achievement 
tests and informal diagnostic tests were examined. Records, pro- 
gress charts, student folders and other relevant documents were 
examined. 

C. Results by Hypotheses 

The quantitative study of the five hypotheses was based upon data 
from the face to face interviews, telephone interviews, paper and 
pencil questionnaires and direct observations. The results of these 
analyses were blended and discussed in relation to the relevant 
hypothesis. 

Hypothesis I 

Sixty junior high school students will participate in an instruc- 
tional program that emphasizes individualization tiirough the modified 
Open Classroom technique. 



Examination of the data to be presented below will show that 



approximately 60 students participated in the instructional program 
at Morse Crisis Intervention Center. The number of students who 
were enrolled in Morse Center at any one time vai*ied according to the 
number of placements received from the D»C. Public Schools Pupil 
Personnel Department and the rate of return of Morse students to 
the regular school program. The findings obtained from the measuring 
instruments showed that there appeared to be strong emphasis on the 
development and use of an individualized learning program for each 
student and that a modified Open Classroom approach was used at Morse 
Center* 

Examination of Table I (Question 10)^ which contains the niunber and 
percentage of students endorsing specific aspects of the program thtst 
are related to the use of an Open Classroom individualized instructional 
approachj, showed that 25 % of the students who responded to 

the paper and pencil questionnaire said that a situation in which 
everyone in class is working on the same lesson at the same time does 
not occur at the Morse Center. 

Only 8 percent of the student respondents endorsed "All of the 
Time" as descriptive of the frequency of occurrence of everyone in 
class working on the same lesson at the same time (Question 10). A 
large majority of the students (47.29i) agreed that "Some of the Time" 
everyone works on the same lesson at the same time. 

Table II presents the number of endorsements and percentage of 
school personnel endorsing specific aspects of the Morse Center Program 
that are relevant to the objective of an "individualized instructional 
program". 

Examination of the data from the responses given to Questions 9 
and 10 showed that 47 % \ of the school personnel respondents said 
that an individualized instructional program was developed for each 
student "All of the Time". An additional 47 % feel that an 

individualized instructional program is developed for students "Most 
of the Time". 

Sixiy-six percent of the school personnel respondents believed 
that the majority of students work on the same assignment at the same 
time only "Some" or "Little" of the time. Thirteen percent of the 
respondents said that this occurs "None of the Time". Since one 
identifiable characteristic of an open classroom approach is the 
difference in ability levels within groups of students and the variety 
of assignments that are therefore appropriate to each ability level, 
the relatively few endorsements given to "same assignment", lent 
further support to the hypothesis that a modified open classroom 
approach was used at Morse Center. 

Additional support came trom analyses of the data obtained during 
face to face interviews with program personnel and student participants. 
Examination of Table III (Items 1, 2, and 3) showed that all of the 
six school personnel interviewed reported that both standardized and 



teacher-made diagnostic tests were used to prescribe amd to indivi- 
dualize learning progra:as for all students ^ Interviews with program 
persomxel revealed that a diagnostic instrument, the Wide Range 
Achievement Test, was administered to the students by the Reading 
Resource Teacher during their first day at Morse Center. The Wide 
Range Achievement Test scores were then used by the Guidance Counselor 
to develop a general program (examples of the programs of the four 
students - whose progress was followed diuring the evaluation are included 
as Appendix A in the Attachment Section of this report) including 
homeroom and academic subject grouping, special interest areas, need 
for remedial work, etc. for each student. These judgements were 
frequently refined by more detailed educational testing with both 
formal and informal tests* Examination by the evaluators of four 
students' folders showed that the Wide Range Achievement Test was , 
indeed administered to these students during each student's first 
day at Morse Center and that test scores were included as part of 
the student's folder. Throughout the school year, the pro j person- 
nel used informal diagnostic tests, observations, and conferences with 
the students themselves to determine whether a student * s ' instructional 
progrdun was appropriate and continued to be appropriate, to his/her 
needs. ... v 

Further examination of Tables III and I, which present the findings 
from analyses of relevant questions from the project personnel face 
to face interview and the student paper and pencil questionnaire, 
indicated that one-to-one tutoring was included as part of the indiv- 
idualized instructional program at Morse. Examination of the responses 
to Item 4 of Table III showed that all of the school personnel inter- 
viewed reported that one-ta-one tutoring was an important part of 
the educational component of the program. 

Direct observations of classroom activities showed that it was r 
not xmusual to find a student working on an individual basis in the 
classroom with an Educational Aide or a Classroom Teacher during the 
class period. 

It was also apparent that Classroom Aides had an important func- 
tion in the classroom setting. Their presence made possible individual 
tutoring during class time. It would be extremely difficult, given 
the demands placed on idie Classroom Teacher's time, to maintain this 
type of individualized program without the direct support provided by 
these Aides. 

One-to-one tutoring was also provided by the Librarian and the 
Reading Resource Teacher. A formal "remedial work*' period with the 
Reading Resource Teacher was often included in a student's weekly 
schedule. For example. Boy C (from the subgroup of students whom we 
followed closely) entered the program with an extremely low (grade 1.9) 
reading level. He received tutoring from the Reading Resource Teacher 
on a twice weekly basis, and was reading on a grade level of 4.5 at 
the time these data were collected (16 school months after Boy C 
entered Morse Center.) 
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Examination of the materials used in the classrooms showed that 
folders for each student containing current work assignments, completed 
papers, and other materials had been established and were in use in 
every classroom. We determined, through examination of the folder 
and interviewing students and teachers , that these folders were used 
to facilitate individualization of classroom activities in the following 
manner: 

Work completed by a student was corrected daily 
and replaced in the student's folder along with new 
assignments. This allowed the student to receive 
feedback about his previous day's work and to get 
new work which built upon his preceding success. 

Examination of the folders of several children in each class 
showed that among the contents of the folder were miineographed work- 
sheets for different lessons (such as alphabetizing, conjunctions, 
etc.) and different ability levels. This lent further support to 
the hypothesis that each student worked according to an individualized 
program that had been developed for him/her. 

Standard time, event, and interval sampling obseirvational tech- 
niques were used to detei-mine whether all students were working on 
the same lesson at the same time during a class period. Table IV 
presents the findings from analyses of the data obtained during direct 
observation of student activities during various class periods 
During six observation periods at least two different activities 
were occurring in each classroom simultaneously. During an obser- 
vation period, it was laical to find a teacher working with a group 
of three to six students, one or two students working independently 
on a different assignment, and an Educational Aide helping one student 
with another lesson. 

The evidence shows that a begixming level Modified Open Classroom 
approach was used at Morse. Direct observations of classroom activi- 
ties, enabled, us^^to^de te mine t 

classroom space and equipment, students seemed comfortable moving 
about or changing seats during class, different lessons were occurring 
simultaneously, and teachers were encouraging students to ask quest-' ons 
and help one another. However, there was little or no evidence of 
learning stations and centers or of an educational technology with a 
comprehensive, graduated and in-depth curriculum as a back up and a 
basis for such stations and centers. For example, it was le^2*hed dui^^ 
our interviews with project personnel that one of the educational 
jgoals was to teach these students how to fill out forms such as 
employment blanks. Although it was apparent that the teachers were 
inventive and innovative in working toward this goal, no stepp seemed 
to have been taken toward building this one series of tasks which 
would permit students to climb the skills ladder. 

Additional support for the use of a Modified Open Classroom 
technique in the Educational component of the Morse Crisis Intervention 



Center was found in the analyses of the responses given by students 
and project personnel to specific questions from the face to face 
intein^iews and the paper and pencil questionnaires. 

Examination of Table I, Question 8, showed that 25% of the stu- 
dents who responded to the questionnaire said that they could move 
to a different seat during class. An additional 44.5% said that 
moving to a different seat duxing class was acceptable to their 
teachers "Most of the Time" or at least "Some of the Time". Thirty- 
one percent of the students endorsed "Little of the Time" or "None 
of the Time" as descriptive of how frequently they were allowed to 
independently move to a different seat during a class period. Of 
this 31%, almost half (5 out of 11> said that it would not be accept- 
able or appropriate for them to move to a different seat at any time 
during class. 

Sixly-one percent of the students (i.e., 22 of 36) said that 
they were able to ask other students to help them with their work 
during class at least "Some of the Time" (Question 9). Four of 
these 22 students said that they could ask other students for help 
during class "All of the Time". Thirty-six percent of the respon- 
dents, however, do not feel that this is something that they can 
do during a class period. 

Examination of the responses given to Question 11 showed that 
students did not report having a great deal of autonomy in pacing 
themselves in their work or moving from one lesson to the next. 
Sixty-one percent of the students reported that they were supposed to 
ask their teacher's permission "All of the Time" before going on to 
a new lesson. An additional 11% felt that it was necessary to do so 
"Most of the Time". If a "Most" of the time response implies that 
the majority of the time they needed to ask their teacher's permission 
but occasionally they did not, then a "Some of the Time" response 
might be interpreted as "it is necessary to ask permission some of 
the time but some of the time it is not necessary." If a "Some of 
the Time" response is interpreted in this f ashi^ ^1?^^?^ 1?^^^^^ 
students said that "Some of the Time" they could go to a new lesson 
without asking permission. Only six percent of the students said 
that it is not necessary for them to ask their teacher's permission 
before moving on to a new lesson. An advantage of freedom- of move- 
ments (physical and in terms of progression to new lessons) is that it help 
students increase their self-discipline and develop responsibility 
for their own behavior. 

Examination of Table V showed that 69% of the students who 
responded to Question 12 in the paper and pencil questionnaire said 
that their work is checked daily by their teachers "All of the Time". 
An additional 5% of the respondents endorsed "Most of the Time" and 
19%, or seven students, of the 36 student respondents said that at 
least "Some of the Time'* their work is checked daily by their 
teachers. Only 5.5% of the students reported that their work was 
never checked daily by their teachers. 
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Sixly-one percent of the st\xdents reported fin their responses 
to Question 13) that they receive daily feedback about their work 
from their teachers at least "Most of the Time", An additional 30% 
of the students endorsed "Some of the Time" iidien asked how regularly 
they received daily feedback about their work. Eight percent of the 
students said that they receive daily feedback from teachers "Little 
of the Time" or "None of the Time". 

Further examination of Table II (Question 14) showed that 4054 
of the participants said that their work with students focussed "All 
of the Time" on helping the students take an active part in their 
own learning. An additional 53% said that "Most of the Time" their 
work focussed on helping the students take an active part in their 
own leaz*ning. 

Fifty-three percent of the respondents said that a Modified Open 
Classroom approach was used most of the time at Morse Center. An 
additional 27% of the respondents said that an Open Classroom approach 
was used "Some of the Time". Twenty percent (all of whom were class- 
room teachers) of the respondents felt that an Open Classroom approach 
was used at Morse Center "All of the Time". None of the participants 
endorsed the categories of "Little of the Time" or "None of tihe. Time". 

The evidence showed that the majority of the school personnel 
who work in a teaching capacity with students felt that a modified 
Open Classroom appz*oach is used most of the time at Morse School. 
The evaluators agree thst many aspects of a modified Open Classroom 
approach were used at Morse Center. However^ as mentioned above^ 
there was no evidence that an educational technology, which is crucial 
to further development*, maintenance, and smooth functioning of an 
Open Classroom curriculum, had been developed. The techniques used 
and thei organization of the instructional program at Morse Center are 
indicators of fine beginning steps in the development of a modified 
Open Classroom curriculum. A more comprehensive, sequentially organ- 
ized educational technology is needed to maximize the benefits to be 
derived from an Open Classroom .approach. ^ _ _ _ . \ 

Hypothesis II 

Sixty junior high school students will be provided with a psycho- 
therapeutic counseling program that will focus on reducing iand/or 
eliminating disturbing behaviorT ■ ' ■■■ 

The evidence to be presented below showed that the students were 
provided with a psychotherapeutic counseling program. A major focus 
of this psychotherapeutic counseling program was the reduction and 
elimination of disturbing behavior. 

Interviews with project personnel and examination of weekly 
schedules (see Appendix B for an example of a Monthly Schedule for a 
typical month) yielded the information that there were several types 
of psychotherapeutic counseling services available to Morse students. 



Several psychotherapeutic consultants provided individual and group 
therapy on a regular basis to selected students and their parents. 
Participation in Guided Group Interaction Sessions (which will be 
discussed in greater detail below), was offered to all students. 
These Guided Group Interaction Sessions were held on a twice a week 
basis and were led by one of the several Morse Staff people who had 
previously taken a course in Guided Group Interaction. 

The School Guidance Coxmselor and the Social Worker also met 
with each student on a regiilar basis to discxiss academic, social, 
and personal concerns of students. The frequency of these meetings 
was determined by each student's individual needs. In addition, the 
Project Director, the Guidance Counselor, and the Social Worker were 
available for crisis intervention. Direct observations of the ways 
in vrfiich several crisis situations were dealt with by these persons 
indicated that sufficient support was provided to the student and 
assistance was given in helping him/her to explore other alternatives 
and feelings. One example of such good intervention by the Project 
Director involved an incident in which Girl A (one of the students 
whose progress we are following in some detail) was feeling very 
"new" at Morse and as a result felt very unhappy. The Project 
Director calmed her, listened to her, and helped her gain familiarity 
with the school and thus increased her comfort level. She did this 
by talking with Girl A for a short period of time and then enlisting 
her assistance as a temporary "office aide". 

A demonstration of a Guided Group Interaction Session was obser- 
ved. It was a "demonstration" session in that the session was open 
to observers during an open house week and in that the session took 
place in some special room in the school. Otherwise, the session, 
its membership, its procedures, etc. were as usual as the participants 
could make it. Direct observation of this demonstration as well as 
interview information obtained from program participants revealed 
that emphasis was placed on the areas of self-concept, empathy, and 
sensitivity to self and others. The primary focus of the Guided Group 

Interaction Sessions was i^^^ reduction and elimination of disturbing 

behaviors. The ground rules set up for the group were:" (1) respect 
everyone's opinion and (2) do not "beat up on" each other physically. 
The groups were based on the assumption that yotmg people pay more 
attention to what their peers say than to what adults say, and dis- 
cussions with peers is therefore extremely helpful. Each group 
ranged in size from 4 to 12 students and were formed by grouping 
students vrtio shared the same homeroom. The sessions were 45 minutes 
long. The group leader had an agenda, or a topic focus, that he/she 
maintained. Some of the topics, however, were suggested by the 
students themselves. 

From interviews with school personnel, direct observations, and 
reading of relevant literature concerning the Guided Group Interaction 
Sessions it was learned that the theoretical assumption made was that 
quite often the students cannot talk about their problems with their 
parents. It was helpful to them to discuss such problems with peers 
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and to discover trtoE o^er students have similar problems. The group 
experience can tbasrafGee be thoug^it of as training in talking together, 
communicatii^ tfaiq^s^ and feelings more effectively, and learning how 
to think in -s^ArasztsifSiil way about oneself and one's actions . There 
was concern not only that the students get content, but that they were 
able to communicate their feelings about that content. The structure 
and philosophy of the Guided Group Interaction approach seemed to be 
consistent with the goals of the Psychotherapeutic Component of the 
Morse program. 

The Demonstration Guided Group Interaction Session led by a 
skilled leader was observed. The group contained six members (five 
boys and one girl). The session took place with the members seated 
in a circle on comfortable living room furniture. Four of the six 
members spoke quite frequently in the group; two members spoke less 
frequently. In the beginning of the session most of the communication 
went through the group leader. However, by the end of the session 
the group members were communicating directly to each other. The 
session was definitely guided, and this was done by the leader in a 
very skilled way. He started with a warm up exercise in vduch the 
members were asked to think of a cartoon character that they like 
and give reasons why they like that character. They were then asked 
to choose what "disguise" they would choose if they could be any one 
of thie characters named. Some examples of students* choices, with 
the reasons chosen are: (1) Roadrunner (a cartoon strip and TV 
character chosen because he is fast and it is difficult to catch him) 
and (2) the Flintstones (also a cartoon strip and TV characters 
chosen because they are comical and sometimes get into trouble Just 
as people do in real life). 

The discussion in this demonstration group session moved to 
several school incidents that involved inappropriate student behavior. 
The group, after discussing the incidents, decided that jealousy was 
one of the causal factors of the incidents. 

The agenda for the demonstration which was^ observed ^^^i^ 
below: 

(1) Warm up exercise (focussing on identifying issues). 

(2) Discussions of current school incidents^ with expression 
of feelings about the incidents and positive resolutions 
of these incidents vdiich were achieved by the group. 

(3) Discussion of past behaviors in contrast with current 
behaviors, discussion about returning to their former 
schools ax^ what that would be like for them. The 
fk)llowix|g, are examples of questions raised diu*ing the 
caourse Ite discussion: "How are you going to handle 
Iming i«SS 35rour old friends?" •'What are your e;qpecta- 
tions «iEi «ieirs?" 

(4) A closdng statement by the leader concerning the con- 
tinualion of this discussion at the next session. 
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In general* observations revealed that the session had clemc 
straictxire and focus, which were perceived not as limiting but am ±n^ 
creasing the value and learning possibilities of the session. The 
topics appeared to be discussed in a productive manner, vrith a sense 
of continuity and responsibility that appeared to be helpful to the 
members. The group leader's closing remarks about continuing the 
discussion and his description Of the agenda for the next meeting 
were examples of such continuity. Another example was the policy 
of having each Guided Group Interaction leader fill out a log record- 
ing his/her observations of the events that occurred during each 
session. Appendix C is an example of the log used by Guided Group 
Interaction leaders. 

Table VI presents the ntanber and percentage of school personnel 
rating the frequency of occurrence of specific aspects of the psycho- 
therapeutic counseling program at Morse Center. Fifty-seven percent 
of the 17 school personnel who felt that they had sufficient eaqperi- 
ence with or information about the Guided Group Interaction (GwG.Ii) 
sessions to respond to the question (Question 25) said that they 
thought that ••the majority of students enjoyed amd benefii^^ 
participation in G.G.I, sessions *'Most of iiie Time'V. 
percent of the respondents said that the students enjoyed^^^i^^ 
fitted from the G.G.I, sessions MSome of the Time•^ The 
of school personnel respondents said (to Question 24) that they 
believed that the issues dealt with in the Guided Group Interaction 
sessions focussed on helping students understand and cope with their 
own behavior at least ''Most of the Time'*. 

There appeared to be a slight difference in the wa^jr in which the 
school personnel respondents perceived the services provided by the 
psychotherapeutic consultants who provide individual service and the 
way in which they perceived the services of those who proyidcj group 
psychotherapy to students. More (68.4% to Q. 28 as opposed^ to 57.9% 
to Q. 29) respondents seemed to feel that the services provided by 
the psychotherapeutic consultants who work with children on an in- 
dividual basis was more often 'in accord with the needs of the stxidents 
aSd'liat 

group consultants. 

Table VH presents data obtained from analyses of thensasasponses 
given by students to questions relevant to the Guided Grot^^ ^litei^ 
action Sessi^s. Examination of the table showed that 41»I3tof the 
students (on Q. 24) said that the issues discussed diiring: ^ift Group 
Sessions we« helpful and of interest to them "All of theBSme'** An 
additional lfi.7% of the students endorsed "Most of iiie lisag^'w^ 
asked by the same quesfeipn if the issues discussed were hri^Eil *o 
them. How^Kcr, 11% of tke students said that the issues aSBsamsBed 
were helpfui^ only a "Little of the Time" and 8% felt that -aK3BBues 
discussed WKpe never helpful to them. The majority (80.68^ li^^ 
students said that at least "Some of the Time" the time spmfeiimifche 
Guided Groiap Interaction Sessions was helpful to ±hem. There raEre^ 
approxima'tely 19% of the respondents (7 of 36) wbo did not endassse 



the session topics as helpftil at least "Some of the Time". Further 
examination of the table (Q. 25) showed that approximately 19% of the 
students did not feel comfortable talking about what was on their mind 
in the Group Sessions at all or only on an infrequent basis. It 
seems likely that there is some overlap between each group of students 
who responded with a '^Little of the Time" or a "None of the Time" 
response to these tw© questions. There are several possible reasons 
for this. If a studicsit does not feel comfortable enough to partici- 
pate in a discussion^ it might be difficult to divorce hifflself /herself 
from these feelings of xineasiness enough to benefit from the discus- 
sion of others. On Oxe other hand, if a student feels that a dis- 
cussion is without value and of no interest he/she would not be likely 
to contribute to it or share his/her feelings or thoughts. It is 
interesting to note that when asked (Q. 36) Aether they enjoyed the 
Group Sessions, approximately 17% of the students responded with 
"Little" or "None of the Time". It appears likely that again there 
is overlap in terms of students who chose "Little" or "None" for the 
questions dealing with comfort level (Q. 22) and perceived helpful- 
ness of topics discussed (Q. 24) with students who enjoy the Grotap 
Sessions "Little" or "None" of the time. The similarities between 
the percentage of student endorsements that fall into the categories 
of "All" to "Some of the Time" for each of these three questions 
support the belief that although a small group of students did not 
feel comfortable im the Guidance Group Interaction Sessions and said 
that they enjoyci: ±t little or not at all* the majority (approximately 
80%) of the studesrts enjoyed the group sessions, felt they benefitted 
from the topics discussed, and were able to talk about what was on 
their mind during the Group Sessions. 

Similar findings were obtained from face to face interviews con- 
ducted with project personnel and students. The availability to 
students of Group Therapy, Individual and Guided Group Interaction 
counseling services was endorsed >/ all the prpject personnel inter- 
viewed. One-half of the students inrterviewed reports that they en- 
joyed the Gubied Group Interaction: Sessions. The remaiiider of the 
^students_ . 
did participate in tiiem: on a regular hasais. When asted abouF^ kiiiSs 
of Issizes discussed during the Guided ^^.oup Intieractixin Sessions^ 
students mentisDned the zS)l lowing areast:.^ii£ighting", ♦!grades", *'going 
to a neif schoaoi", "drug^", "stealing", and "truancy". 

Table vm preseniK the number ai^ percentage of students ex- 
pressi»g piositive fesfts&ss ^toward school* Examinatiomrof "the responses 
to Unestion 33 (I '^jgsste±rnta tirauble" less here at Morro than I xdid 
at nny aast school) saiB»ed t^iat 61% of tie students saifi tha* the 
staieiMnt was true Sar ±hem air leasit ^'SSfast" of "the time. A]tobst half 
or:215 oof the 36 studfents ^o respondedttl;o this guestim, sai 
the^ statement "I get into trouble less^t Morsedthan jtf: my 1 
was true for them "jflil" of the time. These findings ^ient suppo^^rt 
the achievement of the program objective of "a reduction iaat or elimi- 
nation of disturbing behavior". However (and this is^^also japplicable 
to many of the achievements of the program) it is impaissible to 
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determine the causal factor(s) in this reduction in disturbing behavior. 
It may be due to participation in the psychotherapeutic coimseling 
program, but the lack of control and comparison gxaiups precludes attri- 
buting it to a specific program activity or even to the set oF activi- 
ties entitled 'Troject Advance**. Further discussion of this limita- 
tion in the original design of the evaluation of this project will be 
discussed in the Conclusions Section of this report. 

Hypothesis III 

Sixty junior high school students will participate in a program 
of enriclment activities that include sports activities, opportunities 
for career development, and art, music, and drama activities. 

Examination of the data to be presented below showed that all of 
the students who attended Morse School were provided with the oppor- 
txmity to participate in an enrichment program that included a variety 
of recreational, cultural, and educational experiences. 

Interviews with project personnel yielded lixe information that 
the students, accompanied by the majority of the teachers, went on a 
field trip every Wednesd^ morning. Among the activities that the 
siatdents participated in were sports activities such as 6o-Karrting 
and Students vs. Teachers Volleyball games as well as tours of places 
of historical and cultural interest in the Metropolitan Washington, 
O.C. area. Visits were made to the U.S/ Naval Museum, the National 
Zoological Park, the Main Post Office, the FBI BttSklding and other 
places where the students would have the opportuniiy to observe and 
spook with persons who are working in a variety of different jobs. 
A ISst of the 1975^76 School Year Schedule of Field Trips and Activi- 
ties±for the Morse Crisis Intervention Center is included in this 
repocx as AppendrxiB. ExamiTOtion of this schedule showed that 
adtfnrfonal enrichment actiJEiities, designed to facilitate parent and 
cosBsmity involveffiient::as "iiellL as to bring the Morse students into 
c an tor t with scfamls looAs^ near Morse Center, were ^Id regularly. 
Ca^^snashes^^ bake^alest iparties to which female s^sts from a 

lEegiito" isasis dxari-ng iiie lSFS-76 school year. Theas parties served 
^T^nm^^' fflWT^pfTff ^ . t^-HirWrhgg tmrt nnly had a chance to ;broaden their 
u-r»>^i^T > ?f^4-p^ iTiigTO^ ynag their social skills at the parties, but 

xfaerp aarfci es ser^eed as laariiBbrcers for appropriate scholastic behaviors 
iinrrshafc i±e studserits «ces!t^le to "earn" the right to aStend each 
socsbL aKvent . Hor exsssgSjiR^ a Bunny Hop was sponsored by Morse Center 
duriaiE^iApril of 1976. S^ests from Shaw Junior High Schoool were in- 
vr2ae& dancing contests^, r^r^hments, and other activities of interest 
toTfesE STtudents were pSfeammi. In order to attend tlie Bunny Hop, each 
MQi2SfiSi;iCenrter student laped^rbo earn, through good attendance and 
ptjjBE±nali±y, a spec i f iic: mmafae r of points by the day^of the Btinny Hop. 
PoKiaears advertising the Bmmy Hop and progress charts showing each 
student's points were displayed throughout the Morse Center. It 
was JLeamed that almost all students earned 

the^^ivllege of attending the Bxmny Hop. The inclusion of such 
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activities that are contingent upon appropriate social behavior and 
academic effort appeared to be characteristic of the Morse program. 
It seems highly likely that an increase in such contingency-based 
activities would result in even greater increases in students' 
achievements • 

Table IX presents the number and percentage of students endor-* 
sing specific aspects of the enrichment progrem. Fifly-eight percent 
of the students reported in response to Question 17 that they looked 
forward to the Wednesday field trips "All of the Time". An additional 
16% of the students selected *'Mcst,of the Time" ^en asked whether 
they enjoyed the field trips • The Wednesday morning excursions appeared 
to sei^e as liearTiing experiences for the majoriigr of the students 
(Q. 18>. Approximately 70% of the respondents said that they learned 
new things at the places they went to on field larips at least "Most 
of the Time". There appeared to be some career development oppoiv 
tunitifis built into these trips. Forty— one percent of the students 
in reSDonse to Question 19 endorsed "Some of the Time" when asked if 
field itrips gave them a chance to see and meet people vdio woi3c at 
different kinds of jobs, 30% endorsed "Mos* of the Time" and 22% 
endorsed "All of the Time". Interviews with project personnel yielded 
the information that many of the opportunities to meet people who 
work in <Hf f erent jobs were serendipitous opport\mi±ies - that is , 
the primary purpose of a specific excursion was notrto increase ex- 
posure to various jobs and persons who work in different professions. 
Perhaps the number of field trips planned specificauLly for this pur- 
pose might be increased^ An example of such an excsBorsion was the 
field trip to the Post ^fice, which allowed the ^fcsadents to^ Tl^^ 
about the processing lyf mail and to observe people ^o work at various 
jobs in Idie po^tett facaSlity. Informal excursions to nei^iborhood 
businesses iwere amotbecr^ way that the Morse School provided career 
development opportunities for it£ students. Interviews wi±h persons 
from a Local business establishment (i.e. a neighbaateod^easi^^s) 
yielded flie information that Morse personnel f requenSy tbiMUght 3 
groups of students tiver for a tour of the cleaning SKJilxty and jits 
.operations*,- .. . , . ........ — 

Furrt±Der exasffiswaoir of the data presented in Tabae EC ^wed 
that spoitts ;act x s3 a :!::3g s msre frequently scheduled (Q. 20). I± can be 
seen that -41% of ia» stndents responded to Question ao of the paper 
and pencil atjuestionmiEe: by endorsing "Some of the Tiane" as Sescrip*- 
tive of tiiB frequency wfcth which the field trips gave^xthem a chance 
to do sonaB sports actimsfcies. Thirty-six percent of ±he students 
said thair the field txngis provided them with such an !i5>portuniiy 
"All of i±[e Time". 

One of the major reasons why sports activities s«idi as Smisatm^.. 
and Stuctenife'^Ttocher \l^li^b^li^ Basketball 4games were^iheld:^ 
locations^ otiWEr tifflB^^rse Oeaater was tha±: Mixnse GeniOT^^^^t^^ 
a gyiiBxass^imb ^ gw^iPiR^ p^^^ j other p^grsical eduoastioisal facilit^^s^. 
The staiiftEEts^^ thecefc^ tot^ outside of the Cenfer in order to 
swim oir^cs^s^ ba^KtbaXl.. As a^!^ the opportunsfeies to participate 



in such activities were usually limited to days when the field trip 
activity scheduled was a sports activity • 

It appeared that the students did not have much input into the 
choice of vrhere to go cm field trips. Seventy-five percent of the 
respondents reported (on Q. 21) that they helped choose some of the 
places that they went to on field trips "Little" or "NLone" of the 
time. 

Table X presents the nmiber and percea-tage of school p^sonnel 
endorsing specific aspects -sst the enrichment: program at Mon^ Center. 
Ninety-'four percent of the ssespondents (on Q. 20) said that the 
majority of students enjoyeS and benefitted from participation in 
field trip activities at Isaest '*Most of. the Time**, with one idxird 
of these respondents (6 out: of 18) endorsii^ the field trips as 
enjoyable and beneficial ts students "All of the Time". In Table XI 
SA,% of the respondents saiS that (on Q. 23) the field ±Eips provided 
the students with opportxmiities to learn social and coping skills 
necessary for a variety of laituations encountered in daily life at 
least "Mo st^ of the Time". The respondents were not qui^ aa enthusi- 
astic in their endorsements'^ oi^ the field tx%is as proviiiSias^^he 
students with opportunities cabtain izs^^DOwtion about: various 
careers, although 31% of the school perscnnei. who re^pmdssd to this 
item (see Table IX for 22 of the paper -ang pencil tgmsstiortmmTe) 
said that such an opptortunxtt^y was avai^ble *VliDfst of^ i^ie :Time?^ 
However, 47% of the re^ond»ts ±elt iSiat cmfieer devgR^iapmgat oppor- 
tunities were built into the fia3)d trigis :an£7 *^Somer:coiE^e Time", 
and one person (or 5.3% of Jfe# aygsponfefifes) :fielt thaet^araer devel- 
opment opportuni±ies wm^ hmiJL± imto ^ 

of the Time. Caxaeer de^elopsamrt s ar^ i mgtf;y r±ant :::arggt.i3i wwking 
with the student::popiilata.oH Moicse JS^^ AdditfmrnaOl esiphasis 
in this area viocaEld fiftT»vg%^ Lim»rp ' iiMf> m alT^gfgrdy fine^pa^graBs of 
enrichment actibdties. 

Art and cmsic actrvities^^aad cla^ras^mce strongly enahrsed 
(Q. 18 in Tablie X) as a part rnr" "the Murae Center pro^gEam. ^ghty-- 
nine percent of the school pessaonnel reported that such oj^waxrtunities 
Were available to students "Ail of the Time" as a part of iSite school 
curriculum. 

Observation of selected activities^ that occurred :ass p^rt of a 
May Week Open House lei Idie^ evaluatodCE to conclude thsEr there is a 
pervasive attitude of '^•aapwaae^" ahmut tte program and ^itsi x 
that facilitates not om^' igEscussiaai aMdssharing amDng^^sffii^gram parti- 
cipants but also parentrsmfe community Tifmaiinement. Dtmrxj^ddie week 
of May 10-14, 1976, MoTrrorgehaol spsnsmsdz numerousL^OpwEtiMause activi- 
ties, including a garags^ isale , a heaiifcsxmeening, a :pKEtJLiick luncheon, 
and a guided group int raiMdt i tion dem ons l i raal zbom. The ;ob;^ctiv^es of this 
May Week, as describedriai :±ixe mimeogra^md Syer sent ^to^parents and 
posted in the school, w3KEje!:d:o: (1) acmaintr the schomlsxand community- 
at-large with the Morse :Cris is IntervesEtion Center anc: its program; 
(2) give the families of Morse studentsiandrthe commiiffiElgp~at--large an 
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opportunity to join in with the students and staff at Morse in educa- 
tional, recreational, fund raising, and other beneficial activities; 
and (3) provide interested educators in the city an opportunily to 
observe some aspects of behavior management techniques at Morse, 

This belief in the value of sharing of ideas and coox^iination 
of effort was demonstrated by the regular scheduling of 'Home and 
School Interaction Team" meetings. Parents were contacted via mail 
and telephone and invited to attend parent-teacher meetings that not 
only helped familiarize them with the Morse program, but helped them 
to learn new skills in parenting and in the provision of suppozrt for 
their children's good work at Morse. Attaclment 5 is a copy of the 
agenda from one of the 1975-76 school year Morse home and school Inter- 
action Team Meetings. This particular meeting focussed on familiar- 
izing the parents with the psychotherapeutic program at Morse Center. 

Table XV presents some finding derived from analyses of the 
data relevant to the enrichment component obtained from foraal face 
1:0 face inteiviews conducted with four students. It can be seen that 
:all of the students interviewed reported enjoying the field trips. 
TThree of the four students said they did ''sports activities" while 
on iSie field trips. One student said that he had learned about 
different kinds of jobs as a result of the field trips. Two students 
endorsed "going to museums and the zoo", and one student replied 
that he did not know what kinds of things he had learned by going 
on i3.eld trips. However, this same student, although not able to 
verbalize what it was that he had learned while on the field trips, 
reported enjoying the field trips. Informal interviews with several 
other students yielded similar kinds of findings. All the students 
liked participating in the field trips and other enrichment activi- 
ties^ They seemed to feel that some of the "highlights" of the 
year's enrichment program were the Go-Karting trip, the Student- 
Teacher ball games, the visit to the National Zoological Park, and 
the Bunny Hop. 

In general, the enrichment component of the Morse Crisis In^^ 
vention Center program seems to have been well designed. It empKa- 
sized a variety of recreational, cultural, and educational experiences. 
It seemed to be one of the strengths of the program, since it was one 
way of "making learning fun" for students who entered Morse Center 
with a history of failure in academic performance and interpersonal 
relationships . 

Hypothesis IV 

All i?rogram activities will focus on the objective of preparing 
the student participants to return to the regular public school system. 

One of the ways to prepare students for return to the regular 
school system is to help them modify behavior that is inappropriate 
to functioning successfully in that school system. In order to do 
this, it is necessary to help students understand their behavior and 



••cope" with their emotions and impixlses in a mamer that is not 
counter-productive to their own mental health and their success in 
school and later life. Helping them to "cope" would take the form 
of providing them with opportunities to leam and practice new behaviors 
that will prove to be more congruous with success in school. 

The data to be presented below support the hypothesis that all 
Morse Center program activities focussed on the objective of preparing 
the student participants to return to the regular public school system. 

Table XI presents the number and pei*centage of school personnel 
rating the frequency of occurrence of program activities related to 
the objective of preparing the student participants to return to the 
regular public school system. Examination of the responses to Quest-* 
ion 15 of the paper and pencil questionnaire showed that 47% of the 
school personnel respondents said that their work with students 
focusses on helping students understand and cope with their behavior 
problems "All of the Time". An additional 21% of the respondents 
chose •'Most of the Time'^ when asked how often their work focussed on 
helping students understand and cope with their own behavior. The 
majority (68.4%) of the school personnel said (on Q. 24) that the 
issues dealt with in the Guided Group Interaction Sessions focussed 
on helping students understand and cope with their own behavior. 
Eighty-four percent of the respondents (16 of 19 school personnel) 
reported (on Q. 23) that the enrichment component (in particular, 
the field trips) provided the students with opportunities to leam 
social and coping skills necessary for a variety of situations en- 
countered in daily life at least "Most of the Time", of these 16, 
five respondents said that such opportunities were available "All 
of the Time". 

Direct observation by the evaluators of a Guided Group Inter- 
action demonstration supported the endorsements by teachers that the 
issues dealt with in the Guided Group Interaction Sessions focussed 
on helping students understand and cope with their own behavior. 
During the demonstration session obseryed , emphasis wa 
the areas of self-concept, empathy, and sensitivity to self and 
others. The primary focus of the Guided Group Interaction Session 
seemed to be the reduction and elimination of disturbing behavior. 
Among the issues discussed were current school incidents that in- 
volved inappropriate student behavior, past behaviors in contrast 
with current behaviors, and feelings about returning to former 
schools. (Further discussion of the Guided Group Interaction Sessions 
can be found above in the Results section for Hypothesis II,) 

The objective of "preparing students to return to the regular 
school" implies that the students are ready to return to a regular 
school. "Readiness to return to school" is a concept that needs to 
be defined operationally - that is, in what ways does a student's 
behavior need to change and what behavioral goals must be achieved 
before the student can hope to be successful in a regular school 
program? 
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Interviews with project personnel yielded the information that 
the criteria necessary for a student to be considered by Morse School 
personnel as "ready for return to a regular school" were improvement 
in academic performance and social behavior. During the 1975-*76 
school year data obtained from teacher observation^ attendance records^ 
and pre and posttest scores were used by project personnel to assess 
a student's readiness to retxum to a regular program. During the 
first and second years of the project, additional sources such as 
the Pupil Behavior Screening Instrument (D.C. Public Schools, 1972) 
and the '"How I See Myself" questionnaire (a Likert-scale type question- 
naire developed by Ira J* Gordon of the Institute for Development of 
Human Resources, University of Florida) were txsed to obtain data for 
assessment of students* progress and readiness to return to a regular 
program. One of the major reasons why such instruments were not used 
during the 1975-76 school year was that the project personnel felt 
that the additional work involved in responding to such questionnaires 
was too demanding and of an aversive nature to the students and thus 
might be superfluous since it might be possible to obtain data in a 
way that does not involve as much work for the students. The project 
personnel said that they planned to explore this issue further. 

Further examination of Table XI (Question 40) showed that 5% 
of the respondents said that the criteria by which a student was 
identified as ready for transition back to a regular school program 
were clear and well-^iefined "All of the Time". Forty-seven percent 
of the respondents endorsed "Most of the Time", and 31% of the 
respondents endorsed "Some of the Time" when asked to respond to 
this question. It appeared that the majority of the school personnel 
felt that the criteria for "readiness to return to a school syst^^m" 
could be more well-defined. The evaluators also feel that there 
needs to be some kind of standard gtddelines for categorizing a 
student as "ready to return to a regular school", perhaps the estab- 
lishment of specific amounts of improvement in specific areas as 
criteria would serve to make the decision process easier and more 
reliable. 

Only fifteen percent of the respondents reported (on Q.: 41)' 
participating "All of the Time" in the assessment of each student's 
readiness to return to the regular school system. An additional 26% 
of the respondents said that they participated "Most of tihe Time" 
and 15% said that they participated "Some of the Time". Seven of the 
nineteen persons who responded to the questionnaire felt that they 
could not answer the question since their job responsibilities did 
not involve the area of student assessment. It is interesting to 
note that one regular Teacher, one Social Worker, and three Teacher- 
Aides were among the seven who felt that it was a task that was not 
appropriate to their role at Morse Center. It seems likely that 
these people, who interact so frequently with the students, woiild 
have had much to contribute to the assessment of students* progress. 

It was learned from interviews with school personnel that the 
process by which a student was assessed as ready for return to a 



regular school as well as the procedures involved in placing him/her 
were often long and frustrating for school personnel, the student , 
and the student's parents. The first steps involved a Morse staff 
conference at Whicii time the progress of those students who were en- 
dorsed by teaochers as having made substantial progress in acacfenric 
ani£ social behamnr «as discussed. Additional observations of^liiie 
stiiwtents' behaviaar nwre then made before a decision was reached by 
the Project Direciisor and the Guidance Counselor. If they f elt tixat 
tim stxident was reaB«^ to rstum to the regular school system, they 
reeaommended to the »5>ii Persoimd Office that a particular student 
wasi considered by ttSai^ Morse staff to be ready for return to a regular 
sdnool. At this gwrnrty any further decision-snaking as well as ad<U^«-^ . 
tix>nal psycholii3gi2SiBiL^ttesting became the responsibility of a cominittee 
comprised af OTppeaaastatives from the Departments of Pupil Personnel, 
Handicapped Se3:^ica5Si, Placement and other D.C. Public School Offices. 
If the decision of nliiis committee was to place the student back into 
the regular school 3fstem, the placement was made without consultation 
with Morse :School pMssonnel, the parents of the student, or the stu- 
deztt himself* As a result, students were sometimes sent back to the 
school from which tliiey came. This may be tindesirable sioice it places 
a child back into air environment where he/she may be viewed not in 
terms of improved current performance but in terms of not-so-gbod 
past periormar«««. l^or example. Boy C (who was one of the stiidents 
whose progress «e fiskllowed in some detail) was identified ^as being ; 
ready to rettirai 1to tihe regular Public School System in Pailt 1976. 
He and his paBssants were notified by the D.C. Schools Placement Office 
that he is tosfc placed back into the school from which he: xame. A 
telephone intearview conducted with his parents by the evaluators 
yielded the iiaa>rmation that Boy C*s parents viewed such a: placement 
as highly unfesri rable in view of maintaining the improved behaviors 
he learned vdfile at Morse. They feared that it will undo i:he progress 
he made to pil^ce him back among social contacts who reinforce his 
inappropriate behavior and teachers who may view him strictly in 
terms of his jp^t behavior. Howevea:', as the placement procedure now 
stands, theizr omly recourse is to ^peal this decision, which is 
often a lengSiy and nonrewarding process. This evaluation team feels 
that, unless ^Sie receiviifig scHod^ 

for the rertuiarbQg student, there are dangers in this procedure. 

Anotker set of disadvantages of the current placement procedure 
are the time cfelays in placements that appear to result from the 
decentralizatiaan of various depar^teents within the D.C. Public School 
System. InefScient or inadequate iriterdepartmental coordination 
concerning ±haE3teps necessary to implement the placement often results 
in a student iaangxiishing in the wrong environment, e.g., in an envir- 
onment that hs^hae has outgrown, rather than prospering in a new 
^environment tlawt is suited to his/her current needs. Data obtained 
lErom both fonnaiL and informal interviews with Morse Center personnel 
showed that Massase Center was considered by all the persons interviewed 
tto be a ''tempoisfflgsr placement" for all students. They described the 
^l^enter as a placsRifor students "who are coming from a regular school 
jand going back toj^a regular school". 
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A Several of the persons interviewed spoke quite strongly of the 
desirability of establishing close working relationships with school 
personnel from both sending and receiving schools « Further discussion 
of steps taken by project personnel to establish such relationships 
will be presented below. 

The evidence obtained from analyses of the interview data supt>6rted 
the hypothesis that Morse Center progr^un activities focussed on^^^^p^^ 
paring the student participants to return to the regular publics sc^^ 
system. However » additional effort needs to be directed toward^^^^^^m^ 
ifyirig the existing placement procedure so that the progress a- student 
makes while at Morse is not reversed because of long delays, an iri^^^ 
appropriate placement, or inadequate training of the receiving; school 
personnel. 

As soon as a student was notified of his/her placement, the Morse 
Center Social Worker initiated contact with the receiving school in , 
order to exchange information and plan for the availability of an 
appropriate support system for the student. Appendix H is an example 
of an introductory letter sent by the Morse Center Social Worker 
the Counselor at the receiving school. A similar letter was s^nt 
the end of the First Advisory Period following the studient's^^^^^^^^^ 
to the regular school system^ The Social Worker, in addition to 
communicating via telephone and the mail, also miade site visits to 
the receiving school to talk with the Guidance Counselor at ^^^^^^1^^^ 
receiving school and to observe the student in class • Interviews 
with the Social Worker yielded the information that one of her major 
objectives in visiting the classroom or home of a student who has 
left Morse Center was to let the student know that the facul^ at 
Morse Center was still concerned and wanted to continue to help in 
any way possible. 

At the end of the school year, the Social Worker developed a 
follow-up report that contains information about the progress of 
those students vAo were returned to the I'egular . school system during 
the course of the previous year. Appendix F, included/in the' Attach- • 
ment Section of this report, is a page from the 1973-74 Fbllow-up 
Report that was developed by the Social Work staff. Information 
about each student that was obta^ the course of the follow- 

up activities of the jpast^^y^^ well as information obtained at 

the time of the final oide-year follow-up contact, was summarized and 
presented in a table format. The result is a brief but comprehensive 
picture of where students who have left Morse School are, and how ^ 
well they are dbiiig, at the^^^^^ one year period. We learned 

f rbm that approx- 

imately 60% of the students who were returned to regular schools at 
the end of the first year of the program were still in school at the 
end of their one year follow-up period. The findings from^\ 
of the data obtained from project personnel yielded the information 
that all the persons interviewed reported that students who are 
placed back into a regular school program received follow-up service 
for at least one year. ^'^ 



leartied from our interviews with school personnel suid our 
examination of follow-*up records for students who were foil owed-up 
during the 1974-75 school year that (a) 60?4 of the students whox had 
been placed back into a regulax school were still attending that 
school at the end of the one year period, (b) that 25% were in some 
vocational program or were working steadily and (c) 15% had dropped 
out. Morse Center had very little information about the students 
vdio had dropped out • 

In general, the evidence indicated that the project personnel 
felt sti^jngly about the in^ortanc 

to a regular school system and that program activities were geared 
towiax*d achieving this objective. 

Hypothesis V 

All members of the staff vdll participate in pre^^ and in^service^ 
training sendnars that focus on effective teaching techniques, b 

All members of the Morse staff were provided with opp^i^i^i^^ 
for in-sejrvrice training in areas relevant to effective teaching in 
general and to the specific objectives of the Project Advance , program^ 

Table XII presents the number and percentage of school' personnel 
rating the frequency of occurrence of pre- and in-service training' 
as part of the Morse Center program. Examination of the data obtained 
from analyses of the reisponses given to Question 31 of the paper and 
pencil questionnaire showed that 73% of the respondents felt that the 
pre-service training received was relevant to the objectives of the 
Morse program at least"Most of the Time". It is clear that the 
school personnel viewed the training received as extremely helpful 
and relevant. We learned from bur interviews with project p^ 
that the majority of staff development opportunities were available 
subsequent to the start of the Project Advance funding period in 
September, 1 973 and therefore could not be described a^^ :"pre-service*' 
training. However, since many of the school personnel currently with 
the Morse Center were carried over from the Boys Junior-^Senibr High 
School program that was previously housed in Morse School , it may be 
that when responding to Question 31 ("pre-service training was rele- 
vant'^ they were referring to training received long before the start 
of Project Advance. However, in view of the fact that we learned 
that many of the training opportunities mentioned in these interviews 
were sponsored by Project Advance, it seems more likely that the 
endorsements listed as pre-service training were actually references 
to Project Advance in-service training opportunities. 

Further examination of Table XII (Question 32) showed that 52% 
of the school personnel said that in-service training and staff 
development were emphasized at Morse School "All of the Time". An 
additional 42% said that staff development was emphasized'T^ost of 
the Time". Interviews with School personnel, as well as direct obser- 
vation activities and examination of relevant documents, yielded 
information that supported these endorsements. We learned that a 



During this time period, the' school personnel discussed the program 
in general t shared their ideas and feelings about specific aspects of 
the program 9 discussed student's progress, and listened to guest and 
in-^house speakers present talks oh relevant issues * The topics dis- 
cussed during these meetings » as well as the Jbhemesof seminars » work*- 
shops f and courses tijt^t school personnel participated in since the 
incepiiiion of the pro^ject appeared to be relevant to effective teaching 
techniques in gene/al and to the specific objectives of Project Advance. 
It can be seen from examination of the responses given to Question 33 
of the paper and pencil questionnaire (presented in Table Xlll^^^^^^^ 
84% of the respondents felt that the issues dealt with during the 
Wednesday in-service training seminars were relevant to their work 
at the Morse Center at least 'Most of the Time ^^ Seven of these six- 
teen respondents said that the in-service training was relevant ••Ail ' 
of the Timtf^. Appendix I included in the Attachment Section of this 
report is a list of the 1975-76 Staff Development Workshops that were 
held as part of the in-service training program for Morse Center 
personnel. Examination of this docvmient showed that the topics sched- 
uled were indeed relevant both to effective teaching techniques in 
general and the specific needs of teachers at the Morse Center. 

Morse School personnel have obtained training in such areas as 
the Open Classroom and Guided Group Interaction. Slimmer workshops 
(sponsored by the Special Education Department of the D.C. Public 
Schools System) /that were designed to train Educational Aides ^^^i^^^ 
development of learning station? and packages were also held at Morse 
Center. 

An area in which the majority of the school personnel seemed to 
feel the need for additional supervision was the role of Guided Group 
Interaction (G.G.I • ) leader. Although all of the persons who served 
as G.G.I, leaders participated in a course in Guided Groupl^ 
(offered at Federal City College by the person ifAo was^ t^ 
Group Interaction Consultant to the Center) , a desire was expressed 
by mariy of the G.G.I, leaders for additional ongoing supervision in 
this area. Further examination of Table XII (Question 27) showed 
that while 57% of the respondents said that they received adeqixate 
supervision in this area "Most of the Time^', 26% of the respondents 
said that they received adequate supervision only ••Some of the Time'^i 
InteJTviews with the Project the Consultant concerning 

the G.G.I. Consultant's specific responsibilities yiel^^^ 
mation that although he observed the groups and gave some feedback 
to the Teacher-leaders concerning their skills, there was little 
opportunity for the leaders to^m^ the consultant on a regular 

basis for one-to-one supervision. Such individual sessions, if inter- 
spersed with training seminars devoted to Guided Group Interaction 
techniques in which all the leaders participate, could provide the 
Teachers with the additional support necessary to maintain and in- 
crease their skills as Group leaders. 

A second area in which additional training for school personnel 
might be helpful is in the use of behavior management techniques. The 
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faculty were already using ^ with success^ some behavioral, techniques 
such as contingent reinforcement of specific behaviors. It is likely 
that increased frequency and sophistication in the use of such pro- 
cedures wou3,d result in even greater successes in their work with 
children. In order to maintain consistency and quality in the use 
of such procedures ^ it would be highly desirable to devote some staff 
development time to this topic • Emphasis on both the theory and the 
application of behavior management techniques would seem appropriate 
to -t^e needs of the program. 

In general^ the evidence showed that there was a great deal of 
staff development at the Morse Center and that the staff development 
opportunities available to the school pei*sonnel were very relevant 
to the objectives of the program. 

Hypothesis VI 

There will be ongoing assessment by project personnel of students' 
progress and of the functioning of the prograun^^^^ a^ whole . 

The findings from the data to be pr^eSented below showed that some 
positive steps w;ere taken ixi the assessmcjnt of s 
of the functioning; of the progr^am as a^ whole .^^^^^ .additional 
emphasis needs to be placed in the areas of stsmdar^ criteria 
used to determine whether a student has progressed, developin^^^ 
ways to use test scores to assess student ' s progress , and developing 
an internal evaluation design that enables tlae evaliiators^^^^^^^^^^^ 
internal or external, to make definitive statements about causal re- 
lationships between findings and specific project activities. At 
this point in the discussion, only the data that is relevant to the 
existence and use of such assessment procedures will be presented. 
At a later point, the findings obtained from our comparison of pre . 
and posttest scores from the 1973-74, 1974-75, and 1975-76 school 
years will be discussed. The reliability, validity, and appropriate- 
ness of the test selected for use in the internal evaluation of 
Project Advance will also be discussed. 

The evidence obtained from face to face interview data, direct 
observations, and paper and pencil questionnaire responses showed 
that ongoing assessment of students* progress was a regular part of 
the Project Advance program. 

It was learned from examination of records and our interviews 
with various School Personnel that the Wide Range Achievement Test 
(WHAT) was administered to each student during the Fall and Spring 
of each school year (or at his/her dates of entry and departure). 
The pre and posttest scores for each student were then used to deter- 
mine whether progress was made. We also learned that informal and 
formal tests in Mathematics, Reading (e.g. the Morrison-McCall tests 
from the D.C. Public Schools Reading Clinic), and Spelling were ad- 
ministered to each student during his/her first few weeks at Morse 
Center and then readministered periodically throughout his/her 
stay. 



Direct observations also support the hypothesis that there was 
ongoing assessment by school personnel of student progress. During 
examinations of student work folders, it was evident that students', 
worksheets were corrected by the teachers and replaced in the stu- 
dents' folders* It was also noted that teachernnade progress charts 
of students' performance were on display in every classroom. 

It was learned from interviews with school personnel that the 
primary sources of data concerning student progress for the 1975-76 
school year were the WRAT scores ^ teacher observations, and attendance 
records. The Morse faculty feel that a good indicator of how well a 
student is doing is his/her attendance record. If a sttuient attended 
school regularly, they considered this regular attendance to be a 
sign of progress since the majority of the Morse Center students had 
a history of truancy. The assumption was made that an increase in 
attendance behavior usually goes hand in hand with an increase in 
positive feelings about school. In order to feel positively about 
school or any environment, it is necessary to succeed in some way in 
that environment. It is hard to succeed in school when attending on 
a sporadic basis* Therefore, it s.eems reasonable to assume that 
regular attendance may in some way indicate student progress. It 
should not, however, be the sole source of "hard" or quantifiable 
data about students' progress. 

Several additional measuring instruments were used by project 
personnel and outside evaluation consultants to assess students' 
progress during the first two years of the project. Among them were 
the ^Uow I See Myself" Scsde (developed by Ira J. Gorden of the Uni- 
versity of Florida) vriiich is designed to be administered to students, 
the D.C. Public Schools Pupil Behavior Screening Instruiaient (de 
to enable the classroom teacher to assess students' behavior), and 
several teacher-snade parent and teacher mini-questionnaires. There 
were several reasons why these instmaments W(ere not used dxiring the 
1975-76 school year. Apparently, the school personnel felt that 
completion of these questionnaires arid sujrvey fonh 

experience to the students whom they felt disliked "committing" them- 
selves in writing, (2) extremely time-consuming for both teachers and 
students, and (3) perhaps not even necessary since it might be pos- 
sible to obtain similar kinds of information in a less time-consuming 
and disruptive fashion. With these thoughts in mind, a moratorium 
was placed on the use of these and other available measixring 
ments (such as a Student Siirvey developed by Commonwealth Leairriing, 
Inc.) for the 1975-76 school year. It was learned during interviews 
with school personnel that they plan to discuss at length the use- 
fulness of such instruments prior to continuing to include tlxem 
among the instruments used to measure student progress at Morse. 

The findings from analyses of the data obtained from the paper 
and pencil questionnaire supported the hypothesis that assessment of 
student progress was a regular part of the Morse program. 

Table XIII presents the number and percentage of school personnel 
saying that various activities relevant to the ongoing assessment of 
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stiident progress occurred. Six1y--eight percent (that is^ IS out of 
19) of the school personnel respondents said in response ±o Question 
16 that they were able to review daily each student's work at least 
'»Most of the Time''. Seven of these thirteen said that they were able 
to review daily each student ^s work "All of the Time". Since four 
people of the 19 school personnel who completed the questionnaire did 
not feel that their particular job responsibilities at Morse Center 
entailed reviewing students' work, the 13 school personnel (or 68% 
of the total group) who endorsed at least "Most of the Time", repre-* 
sent an even larger percentage (13 of 15 endorsements, or 87?l) of 
those who responded to this question. 

Further examination of Table XIII (Questions 37, 38, and 41) 
showed that there seemed to be a definite delineation of job respon- 
sibilities with respect to assessment of student academic and social 
progress. Six respondents said that they knew such assessment oc- 
curred but that tiiey themselves did not participate in assessment 
activities because such activities were "Not AppropH-ate to My Role", 
Since examination of data concerning the roles of the responded 
yielded the information that only two of the respondents served in 
an Administrative Assistant capacity^ one can conclude that some 
members of the Teaching, Guidance, or Social Work staff feeJi'-jdS&t 
assessment activities do not fall into their b^liwick. Oxif^mlt ithe 
assets of the Morse program is its small size^^JBd the resailtSs 
creased qpparrtunities far personal interactrorB and rela^fcionai^s 
between sstaS: and students. Since all personiasel interacted wS3k 
studente on::3a frequent basis, it seems at best inefficieafc^ioi ignore 
the resoBTcsss of all school personnel when assessing stidnifes^ pro- 
gress. Of uaaxirse, certain school personnel Jrole s probaJili^ipaivi ded 
more opposrtas&oities to assess a specific type of student fnpess 
(for exaopke. Classroom Teachers and Educational Aides baauED&equent 
opportundtties to assess academic progress) but it is important not 
to overlook the fact that valuable contributions can be made by all 
members of a staff. 



A larger percentage (47.4%) of the respondents stated that on- 
going assessment of student social-personal adjustment (Question 36) 
occurred "All of the Time'' in comparison with the percentage of 
respondents (36.8%) who stated that ongoing assessment of student 
academic progress (Question 35) occurred "All of the Time". However, 
further examination of the table showed that the percentage of respon- 
dents (36.8% and 47.4%) who endorsed either "All of the Time" or "Most 
of the Time" for academic progress was equal to the percentage of 
respondents (47.4% and 36.8%) who endorsed these same two categories 
with respect to student social-personal adjustment. In effecti a 
comparable number of school personnel said that assessment student 
academic progress and and of student social -personal adjustment oc- 
curred at least "Most of the Time". 

The next discussion will be the assessment of the functioning of 
the program. The evidence to be presented below indicates that there 
was some ongoing assessment of the functioning of the program. 
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However, the evaliwtors feel that a more comprdtaensive evaluation 
design would allow the project personnel to increase the effective- 
ness of their evaluative ^efforts. 

The findings from the analyses of the data obtained from school 
personnel responses to questions relevant to ongoing assessment of 
the fimctioning of the program are presented in Table XIV. Included 
in this table are the responses to questions that focussed on (1) wheth- 
er school personnel implemented procedures to assess the degree of 
correspondence between program objectives and program activities, 
(2) v^ether the program had appropriate and adeqxiate resources and 
equipment, and (3) school personnel's observations and feelings about 
specific aspects of the program and the program in general. 

Examination of this table showed the distribution of responses 
given by school personnel who were asked to evaluate whether the 
psychotherapeutic consultant who worked with children on an individual, 
basis provided service that was in accord with s*udei£ts' 3weds and 
program objectives* Four of the 19 school persramel3>who completed th« 
questionnaire did nriot JSai^ that they 

jcsefspond to this Cjaestiam^ Thirteen of the 15 §87%) peopfe vriio did 
Wfsspxmd felt that theix^^hotherapen^c consuSasmts scroylSed se 
1tiiat::iaas in accord wi:S^:S±udents * needs and pxaEE(gramNi^je:d0.ves at 
Isfflst "Tilost of the Tiiaas^. However, it was alscr leaiarod dtitat^^a^ 
^iSaonaiL communication aaafe^coordination between Jhe psf!:chbi2lBerapeutic 
CQDsultants and the sdiool personnel would be ^^sirsQile iii: order ^^t^ 
the benefits aierived from the servicesiiiof such consul tahts. 
Tsm :of the school perstsmnel respondents said that the service provided^^^^ 
v«s :in accord with the students' needs and the objectives of the pro- 
gram only ''Some of the Time". Four of the school personnel who were 
questioned did not feel they knew enough about the consultative service 
provided to respond to the question. It is evident that the service 
provided by the psychotherapeutic consultant was looked upon favor- 
ably by many of the school personnel, but some school personnel did 
not have knowledge of it. It would seem desirable to familia^^ 
all school personnel with the sei^vices provided and to deyelop some 
guidelines to increase flexibiliiy in revising the exact natiii^e of 
the services purchased as the need arises during the course^ of t^^ 
school year. In this way the benefits derived from gooid consul taents 
can be maximized. 

A second area that was extremely relevant to therfunctionjmg of 
the program was the asawassment by the staff of their^ own ''team^^^^^^^^^e 
Examination of Table fflST (Question 34) shows that IJie re spond^ts 
appeared, to. be enthusisas ti c_ about .jthe ampwnt_af suppprfejmd ^P^^^ 
tion that ekisted amongr staff members. Fifty-seven percent ofrithe 
school personnel endorsed "All of the Time" when asked to describe 
how often they received support and cooperation from other staff mem- 
bers. An additional 36% endorsed "Most of the Time". None of the 
respondents chose the "Little of the Time" or the '»None of the Time" 
response alternatives. 



Table XIV also presents data relevant to how school personnel 
viewed the Morse School physical facility (Question 47) and the 
available equipment and learning materials (Question 48) . The major- 
ity of the school personnel respondents said that the Morse Center 
physical facility was inappropriate for the needs of the program. 
Seventy-three percent of those questioned said that the physical 
facility was appropriate to their needs "Little" or"NoneV of the 
time. However, 21% of the respondents felt that "Most o£ the Time" 
it was appropriate for the needs of the program. Feelings about the 
adequacy of the program *s supplies and equipment also varied, with 
26% of the respondents endorsing them as adequate ^^Stast of the Time", 
47.4% of the respond^fs enpiarsing them as adequajt^ ^ome of the 
Ximgll and 21% who fe3|t thafc^ey were adequate "l££ttie of the Time".. 
It s«MP personnelr.that the general 

fe«Li»g:is that the^^^S^ facility was inadeqiiate in several 

iaipoiitett areas for 1^ One deficit was the 

1»TlmrFs;a gyiinas iumuand ^nsmninjg pool both of vdiich are; resources 
lausiiELy^available in^^ p high schools. As .a result, the 

MiK!SK^s±udents had to to various othei^ facilities in 

cm&esstSor' the 6 "providinMoppontnnities f or 

pm&ff i : bg ation in sports ^^efcLvities" to be met. Sin©e it was nec- 
eaEBiijr^or the sinidents ItKleave the school grounds,^and since ar-* 
I!a^p^pIeEts with other schPcBLs or coinmercial establl^nents for the 
usaesairfaequipment or a facSHty had to be prearrangedxby school per-- 
sanzsel ,^such opportunities were not always • readily assrailable to the 
stu dents . A s<5Cond inade^acy of the Morse facility was its lack 
of one-way observation mixrors, counseling offices, time--out rooms, 
and other items that would have provided support to the psycho- 
therapeutic component of the program. 

It was learned from the Project Director that it is rumored 
that the Morse Center program is to be moved to a new faciii'ty in 
the near future. The proposed site is, like Morse Center, an ele- 
mentary school building. It does not have the nec^^^ 
for the psychotherapeutic or physical ediacation componenibsiP of ^^^^ 
program. One of the major differences bet^'/een the Morse f^hool 
facility and the facility being considered to house the Morse program 
in the future is size - the proposed site is quite a bit larger than 
Morse. An increase in the size of the physical space is riot v^^ 
as an asset. The population which Morse Center serves rrequires care- 
ful monitoring so that the more aggressive student does not harrass 
or harm the more timid student. It would be impossible to prpyide 
tliis kind of monitoring, given that the small size of the faculty 
remains stable, in a larger area. It seems that the Morse facully 
have not been given^ the opportunity ^to give-' input^itito-any^pians^-^ 
that may exist for relocation of the Morse Crisis Intet^vention Center 
program. It would seem appropriate, in fact even highly desirable, 
that these people who are most familiar with the needs of the pro- 
gram be consulted on this issue not only for the relevance of their 
ideas but also for the increase in staff morale. 



Additional data that supported the hypothesis that ongoing 



assessment of the functioning of the program occurred was obtained 
from face to face interviews witik school personnel. Several times* 
during the 1975-7® sclmoi year the entire staff met for discussion 
of their relatioisships and interactions. These group process ses^ozrs 
Were led by the Gmded Groap IiKteraction consultant. Specific issues 
relevant to the SuroSS^nix^ of^^^^d^ program were often discussed at 
the Wcninesday Staff IJtaMlopmen^^^ On June 9, 1976/ t!ie agemia 

for the Staff Developwint meeting was tcomprised of items that per« 
tained to the evalual^n of the program's functioning. Attachment 9, 
is a list of the -topiacs that were scheduled to be discussed that day^ 
The Project Director verified that these discussions did in fact 
occur. 

The Wide Range Achievement Test (^ffiAT) will be discussed in terms 
of (1) its appropriateness for i:he way it was used in the Morse pro- 
gram and (2) its reliability and validity. Data from the i973--74 and 
1975-76 pre and past-best administrations of the WHAT will also be 
discussed. The find£]^s from these data and suggestions by the eval- 
uators for additional 5analyses \d.ll follow. 

The Wide Range Achievement Test has been found to be of value 
in many areas of larowitedge and itheir practical a 
Manual, J.F. Jastak. aid S.R. Jastak, Copyright/ 1965, by Gui^^ 
Associates of Delawarer, Inc., Wilmington, Delaware) . The following 
areas are relevant to the objectives of the Morse Center program: 

(1) The accurate diagnosis of reading, spelling, and arithmetic 
disabilities in persons of all ages, 

(2) The determination of instructional levels in school chii** 
dreh, 

(3) The assignment of chiilftW^ to instrui^tsionai groups 
Hresslim a% siiiyliar i?aites and their transfer to faster 
or slower groups in keeping with individual learning 
rates, and 

(4) The comparison between school achievement and other efiebili-- 
ties-, in. all individuals, especially those who are distairbed 
or maladjusted. 

Since the WHAT was used at Morse Center to diagnose the: r^^ 
math, and spelling levels of students, and to prescribe ap^ro^^^ 
programs suid group students according to ajiility levels^ 
for its selection can be supported. However, becausei the WMT ca^ 
diagnose for specific kinds of skills needed with a^ 

( for example, the WHAT can tell one that ajchilji^ is below e^ectejd^_^_^ 
grade level in reading, but it cai^ 

with short vowels or consonant sounds) , it sh^ used in^^^^ 

tion witti tests that can make these kinds of distinctions. ^^^:^^^^^T^^ 
Morse School personnel insured that such distinctions were obta^ 
when it administered to students the Morrison-McCall Reading Test 
and other appropriate tests. ...... 

Another advantage of the IVRAT is that it can be scored immediately 



by the school personnel wrth a mininm o^ and effooErt. The inter- 
views vdth Jtese personnel yielded ia« infiormatian thatniiiis was among 
the factors considered when they selected :±he IVRAT. Siasxeae their per- 
sonnel resourrres were limited, and since ii; was crucial^^:^^ 
diagnose abilirly levels of each incazming student as sooiraas possible > 
the WRAT was an appropriate instrunrait to select. 

The WRAT satisfies quite adequaefcely :the statisticaELxonditions 
of reliability. The reliability codEfici«nts derivedt&som split--half 
forms of the same test range from .SiZl to .gTS, dependi^lt on the 
chronological age. The authors of l^ie test caution thadt these reli- 
ability coefficients do not necessarSly represent accuDBfeBly the 
clinical reliability of the scores. They^repfort that om±he basis of 
clinical experience and some validiljy; caictxlations, thesmDst i^eason- 
able guess concerning the clinical reliability of theilMKt is an 
average reliability of .93 with a range of .90 to .95^, 

The validity of the WRAT varies according to the^BPdftiods used to 
estimate the validity. The ^VRAT is considered to havessaffSairly high 
validity* It has obvious content validity for the assessment f.pfv\^ 
academic achievements. The three parts of the test are core parts 
of a: s^ curidculum. The items within each partj.are 

sequenced well in terms of difficulty from very easy to very^^^ 
the high p intercorrelations among the three parts of the WRAT 

and other measures of academic achievemeofit istee^uding, ^H|i«oia]^, 
school grades themselte^ proiii^ t^iilihce^^^O^^^^ 
and external vaUdi%tsr mM\ ^ wide variety of validity criteidcai. 

The findings obtained ty Morse School ^personnel frianrdtedr anal-- . 
ysis of the WRAT pre and poBttest data f romicthe 1974-75,2aml:J2975-76 
school years show thatrthe^lto^swe School personnel did campSbetnand use 
these data to assess the pisscrgress of the Morse Center students. The 
way in vrtiich the Morse Schoeii personne^ was to ask the quest- 

ion - \<hat percentage of the total group improved? Examination of 
the IVRAT summary data sheet (a portion of ttiis siammary sheet is in-- 
eluded in this report as Appendix K) showed that the school personnel 
determined that ••98.1% of the students who attended Morse during the 
1974-75 school year improved". 

The meaning of this statement seems somewhat ambiguous. For 
example, does "improved^^ mean that there has been an increase in 
grade level from pretest score to posttest score in all three (Spell- 
ing, Math, Reading) content areas? Or does it mean an increase in 
at least one content area? The criteria for classifying- or consider- 
ing"^ a*^'student-^as"^"improved^^'^must--be--clearly-^defin^ 
must be given not only to specifying the precise number of content 
areas in which an increase in test scores must occur but the size of 
the increase itself must be operationally defined. This definition 
should include a statement that describes improvement as a specific 
amount of academic improvement in years and months increase for a 
corresponding interval of time from pre to posttes ting. Some improve- 
ment, or increase in posttest scores, is to be expected simply because 
of the passage of time. 



Let us as Slime that a student entered Morse School in September 
and was returned to the regular school program in June of that same 
school year. How much of a grade increase in his/her test scores was 
to be expected over that ten month period? A reasonable as sumption ^ 
based on criteria used by the majority of school systems, is a one 
academic year, or one grade level, increase. However, we know that 
in their school careers to date the Morse students did not progress 
at the rate of one grade level per year, otherwise they would not be ^ 
behind academically. An assumption made is that the students (1 ) start- 
ed behind at grade K, (2) progressed slowly from K to current grade, 
or (3) both. A big problem in this discussion which makes calcula^ 
tions very suspect is the type of scale that grade level is. It is 
probably an ordinal scale, with no zero point and no equal intervals. 
However, our argument is still approximately correcty since it is 
necessary to make certain assumptions when trying to analyze data. 
We will assume that a child starts out at the "0" point in terms of 
grade level when he/she starts kindergarten or first grade. It might 
be helpful at this point to present an example of an alternate way to 
analyze the test data that takes into account the student !s past, per- 
formance when determining what V'improvement" meansVfor a particular 
studenti. The amount of grade increase that is "improvement" for one 
studiQWi::^€lpy not be "improvement" for a second student. 



In order to make such an analysis it is necessary to know a 
student's chronological age, his actual grade level at the time of 
the pretest and his grade level scores on the pre and posttest. We have 
such information about the four students whose progress i we have been 
following more intensively at Morse School. The information we have 
concerning "Boy C'will beused as an example of a way . to use these 
test scores to measure student progress. Boy C is 163$ years old. 
(The fact that he has remained in school for J5 year past the age at 
which he could legally drop but is, in the evaluator's belief, an 
endorsement of the quality of the Morse School program and the people 
who are part of it.) H^^ 7th grade prior to rentering^ 

Morse School . He was at Morses from the end of NovembeiS;^J^19^^^ 
Junev 1976, lAich is a total of a little over 1 and 2/3 scKool yeiars 
or 17 school months. His WHAT test scores for 1974-75 and 1975r76 
pre and posttests are~preserited below: 

Reading 

. Diff., , . 

Pre Post Dxff . 

1.9 3.3 +1.4 :.:AAM\.y 





Spelling 






Math 








Diff. 






Diff. 




Pre Post 


Diff. 


Pre 


Post 


Diff. 


1975 


2.2 3w2 


+1.0 
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+1.0 


1976 


2.3 3.2 


+0.9 


3.9 


5.2 


+1.3 



3.9 4.5 +1.6 



since Boy C was in the seventh grade at. age 16. 5. years, and 
since it is reasonable to assume that he isntered Grade 1 atv.age six, 
it is also reasonable to assume that he has made an increase-^t: seven 
grade levels in 10.5 years. This works out to be 2/3 of a grade! 
level in one year. Since school years are comprised of 10 months. 
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he averaged an increase of 6.6% of a grade level each month prior to 
attending Morse Center, We can then say that his average rate of pro- 
gress prior to attending Morse was 2/3 of a grade level each 10 month 
period. Any increase over this amount might be considered"improVement»' 
His 1974 to 1975 pre and posttest scores can then be examined in order 
to compare any grade level increase with the criterion grade liBvel 
increase, (i.e. more than 2/3 of a grade level in any 10 month peiriod) 
that has been established as necessary in order to say that he Has 
"improved". 

Examination of the Boy C*s 1974-75 WRAT test scores shows an 
increase, from the September 1974 pretest score to the June/ 1975 
posttest score, of one school year in his Spelling and Mathematics 
teist scores and a 1.4 school year increase^ in his Reading score. 
Each of these increases is greater than the criterion increase of . 
2/3 of a school year, therefore, we can say that Boy V 
in Spelling, Matheiaatics, and Reading during the 1974-r7 5 schpo 
During this school year , he progressed in all three content ar^sas; at . 
Ja more rapid rate than he had been progresising prior to, coming tb^^ > 
Morse School . Examination of his 1975-76 ^ChooUy^ 
posttest scores also shows that (by the criteria prjssen^ 
he caii be said to have "improved" . In fact , his rate improvement 
for 1975-76 was greater in two of the three content areas^than it 
was for the 1974-75 school year. 

The informal and preliminary analyses of this data ttiat was done 
by Morse School does not provide this kind of detailed information 
about student's progress. In effect, Morse School perspnrielwe^ 
using the information about students ' progress /feat Was avsdls^ to 
them and by doing so, shortchanged the program of credit due it. 
Substantial progress was made in Spelling, Reading, and Math by this 
pairticular student dtoring his first year at Morse, and even greater 
progress during his second year. 

We propose that improvement scores be calculated for each of the 
three content areas (Math, Spelling, Reading) of the WRAT. In this 
way, Morse School personnel can determine^irx. which area a child is doing 
very well in or which area additional work is needed. Also, such 
analyses will provide data relevant to tiie strengths or wealmesses of 
specific components of the academic curriculum. 

Conclusions 

A. By Hypothesis 

The discussion of the conclusions will be organized around the 
empirical hypotheses. 
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Hypothesis I 



Sixty jtmior high school students will participate in an instnzc- 
tionalT'pro^ram that emphasizes individualization through the modified 
Open Classroom technique > 

An individualized instructional program was developed for each 
student within the first few days of his/her arrival at Morse. Test 
scores from the Wide Range Achievement Test and other formal and in- 
formal diagnostic tests were used to prescribe a student *s general 
program and his/her need for remedial or accelerated work in specific 
areas. Throughout the school year, teacher-made diagnostic tests, 
observations and conferences ;fith the students themselves were used 
to determine whether a student's program continued to be appropriate 
to his/her needs. 

One-to-one tutoring was an important aspect of the instructional 
program. One-to-one tutoring was provided by Classroom Aides, the 
Reading Resource Teacher, the Librarian, and the Classroom Teachers. 
The Classroom Aides had an important function in the classroom set- 
ting. Their presence made possible individual tutoring during class— » 
time. It would have been extremely difficult to maintain this type 
of individualized program without the direct support provided by 
these Aides. 

A beginning level Modified Open Classroom approach was used at 
Morse, There was flexible use of classroom space and equipment. 
Lessons on different levels occurred simultaneously, with students 
working on specific leissons according to their ability level. However, 
there was little or no evidence of learning stations and centers or 
of an educational technology with a comprehensive, graduated and in- 
depth curriculum as a back up and a basis for such stations and centers. 
The lack of these stations and centers made it difficult for students 
to work independently. 

The Classroom Teachers and Aides were enthusiastic, dedicated, 
and innovative. A good rapport seemed to exist between them and the 
students . Contingency-based behavior management techniques were occa- 
sionally used in working with the students. Tlieir successfx^^ 
coupled vd^th the special needs of the Morse popxilation, is an e^ 
ment for the development and use of a comprehensive behavior manage- 
ment program that focusses not only on academic skills, but also on 
selfHnanagement and social skills. 

Hypothesis^II — — ' - — — .^^-^^..^ 

Sixty jtmior high school students will be provided with a psycho- 
therapeutic counseling program that will focus on reducing and/or 
eliminating disturbing behavioFT " ; 

Several kinds of psychotherapeutic services were available to 
Morse students. The majority of students participated twice a week 
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in Guided Group Interactions (G.G.I. ) Sessions. The theoretical basis 
for and the structure of these Guided Group Interaction Sessions seemed 
to be consistent with the goals of the psychotherapeutic component of . 
the Mox^e program. The G.G.I, sessions focussed on the areas of self- 
concept » empathy, and sensitivity to others as a way of reducing and 
eliminating disturbing behaviors. The sessions provided the students 
with an opportunity to share and discuss ideas, communicate thoughts 
and.i£eelings more efSsctively, and think in a planningful way about 
themselves and therr actions . 

The G.G.I, leaders (Morse School Personnel vrtio had received some 
training in group cooaseling techniques) appeared to be responsible, 
creative people who Imew how to structure the sessions so that issues [ ^ 
were discussed in a productive maxmer. They need to be provided with 
continued training and supervision in group counseling so that th<^ix; i^^^ 
skills will be maintained and increased. ^> 

A psychotherapeutic consultant provided individual therapy to . 
selected students viho had been identified as needing a more intensive'^ I 
therapy program and whose parentis approved' of their childf.s pfU^ici- ' 
pation in such a program.^ At times the parents or siblings^ ^t'/^f?^^ 
dents also participated in family therapy sessions provided^ jby lt)^ 
consultant. A two person psychotherapeutic team, comprised; of; a , ;\ 
psychotherapeutic consultant and the Guidance Cpiinselor, provided 
group therapy to selected students on a weekly basis. This consultant 
was not paid out of Project Advance funds but rather from Medicaid.;"; , 
Accordingly, one of the selection criteria for participation was 
eligibility of a student for Medicaid payments. The criteria forv 
identifying students in need of the Group and/or Individual Therapy . 
seemed to be an informal assessment by School Personnel of students! 
needs for more intensive psychotherapeutic service* 

The School Guidance Counselor and the Social Worker also met 
with each student on a regular basis to discuss academic, social, and 
personal concerns of the studentsv The frequency of these meetings 
was determined by each student's individual needs. 

The Project Director, the Guidance Counselor, and the Social 
Worker (as well as the remainder of the Morse Staff if the need arose) 
were available for crisis intervention. These persons demonstrated 
a capability to provide support to :a student while helping him/her 
explore feelings and alternatives. 

Hypothesis III 

Sixty junior high school students will participaH:e in a program 
of enrichment activities^ that include sports activitiesioppor 
for career development, and art, music, and drama activities. ~^ 

All students who attended Morse School were provided with the 
bpportiinity to participate in an enrichment program that included: a^^^^^^^ 
variety of recreational, cultural, and educational experiences. 



Every Wednesday morning was designated as "Special Activities" 
Day. The majority of these Wednesday mornings were used for field 
trips, which gave the students a chance to participate in sports 
activities (e.g. Go-Karting, Swimming, Volleyball games), and to 
visit places of historical and cultural interest in the Metropolitan 
Washington, B.C. area (e.g. the National Zoological Park and the U.S. 
Naval Museum). Several of the sites for field trips were chosen 
primarily to give the students an opportunity to observe and speak 
with persons who work in a variety of different kinds of jobs (e.g.^the 
FBI Building, the Main Post Office for Washington, D.C., cleaning 
establishments, etc.). Students seemed to very much enjoy these field 
trips, even though they seemed to have little or no input in the selec- 
tion of the sites. 

Activities designed to facilitate parent and community involvement 
were emphasized at Morse School. A Morse Home and School Interaction 
Team was established and met regularly. Additional activities, such 
as Bake Sales, an Open House, and Health screenings, were also included 
among the parent-community-school liaison component. 

Opporttmities to participate in social activities (e.g. a school 
picnic and Bunny Hop) that were contingent upon appropriate social 
and academic behavior were included in the Morse enrichment program. 
Such contingency-based activities seemed to result in an increase in 
positive behavior. It is reasonable to assume that the continued use, 
and even accelerated use, of such contingency-based activities would 
result in even greater behavioral gains for students. 

Hypothesis IV 

All program activities will focus on the objective of preparing 
the student participants to return to the regular public school system. 

One of the ways in which to prepare students to return to the 
regular school system is to help them modify behavior that is inappro- 
priate to functioning successfully in that school system. In order 
to accomplish this, it is necessary to provide them with opportunities 
to learn and practice new behaviors that are more likely to be con- 
gruous with success in school. The objective of "preparing students 
to retUCTi to the regular school" needs to be defined operationally. 
That is, what behavioral goals must be achieved before a student can 
hope to be successful in a regular school program? Morse School de- 
fined these goals as improvement in academic performance and social 
behavior. The means of achieving these goals were providejd through 
the various aspects of the Morse program. For example, the instruc- 
tional program was individualized to better meet each student's 
academic needs. The psychotherapeutic program helped the student 
increase his/her ability to commtinicate with others, acknowledge 
and deal appropriately with feelings, and think about himself /herself 
in a meaningfixl way. 

The Morse School developed a set of procedures to facilitate 
the adjustment of a student who was returned to a regular school. 



Included among these procedures were conferences with the student and 
his/her parents, conferences with the Counselor at the receiving school, 
visits to the receiving school, as well as additional follow-up contacts 
with the student and the faculty at the receiving school. These follow- 
up activities lasted for a one year period following the placement back 
into the regular school* 

The Morse faculty's lack of input into the decision-making concer- 
ning the actual placement sometimes resulted in placements that were 
not advantageous to the student. For ex^ple, frequently students were 
sent back to the school from which they came. This may be undesirable 
since it places a child back into an environment where he/she may be 
viewed not in terms of improved present performance but not-so-good 
past performance. A second set of disadvantages of the current place- 
ment procedure are the frequent delays in the decision-making about 
placement that seems to result from inadequate interdepartmental coor- 
dination £unong the various D.C. Public School departments jointly res- 
ponsible for the decision and its implementation. The result was 
that a student is left to languish in the wrong environment or in an 
environment that he/she had outgrown rather than prospering in a new 
environment that would be better suited to his/her current needs. 

Hypo t hesis V 

All members of the staff will participate in pre- and in-service 
training seminars that focus on effective teaching techniques. 

All members of the Morse staff were provided with opportiinities , 
for in-service training in areas relevant to effective teaching in 
general and to the specific objectives of the Project Advance program. 

School personnel participated in courses (offered by Federal City 
College and other educational institutions in the local area) in such 
areas as the Open Classroom and Guided Group Interaction. Summer 
workshops » sponsored by the Special Education Department of the D.C. 
Public Schools System, designed to^train Educational Aides and Teach^ 
ers to develop learning stations and packages, were also held at Morse 
School. 

Staff Development Seminars were held at Morse School each Wednes- 
day afternoon. During this time period, the school personnel discussed 
students* progress and the program in general, shared their ideas and 
feelings about specific components of the program, and listened to 
presentations on such topics as ''Mainstreaming" and ''Classroom Meetings 
as a Group Process'' from guest and staff speakers. 

Some supervision relevant to the role of Guided Group Interaction 
leader was provided to the Morse staff. However, additional training 
and perhaps one-to-one supervision from the Guided Group Interaction 
consultant would be highly desirable as a means of maintaining and 
increasing their skills as Group Leaders. 

Training in the use of behaviorally oriented teaching and behavior 
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management techniques would providi.:> the school personnel with the 
skills necessary to develop and implement contingency-based, sequen- 
tially ordered academic and social skill programs for students* 

Staff development was strongly emphasized at Morse School. The 
staff development opportronities available to the school personnel 
were very relevant to the objectives of the program. 

Hypothesis VI 

There will be ongoing assessment by project personnel of student's 
progress and of the functioning of the program as a whole. 

Some positive steps were taken in the assessment of students' 
progress and of the ftmctioning of the program as a whole w Additional 
emphasis needs to be placed in the areas of standardizing criteria 
used to determine whether a student has progressed, developing new 
ways to use tC3t scores to assess students' progress, and designing 
a comprehensive internal evaluation plan that allows the evaluators 
(both inside and outside) to make definitive statements about causal 
relationships between findings and specific project activities. With- 
out such statements the D.C. School system has no basis for deciding 
to teiiuinate, modify or continue a program or any of its aspects. 

A formal diagnostic test (the Wide Range Achievement Test) was 
administered to each student during the Fall and Spring of each school 
year (or at his/her dates of entry and departure). These scores were 
used to help build a general program for each student and to assess 
his/her progress. Additional tests of both a formal and informal 
natiire, wei*e administered to each student during his/her first few 
weeks at Morse and then readministered periodically throughout his/her 
stay. 

Attendance records were also used as a source of data about 
students' progress* ^"ftiile an increase in attendance can in and of 
itself indicate that a student has progressed from non-attendance to 
regular attendance, it is important to use other sources (i.e. tests, 
observations, etc.) in order to determine whether progress in specific 
academic or social areas has occurred. 

The method used by school personnel to analyze data derived from 
the Wide Range Achievement Test (WRAT) was perhaps too informal and 
preliminary to provide sufficient information' about students' pro- 
gress. It did not make use of all available data. An alternate way 
to analyze the WRAT data was presented in which the evaluators exam- 
ined the test scores of a student who had been at Morse almost two 
years* The analysis of his test scores showed that he had made pro- 
gress in all three content areas of the WRAT that was greater than 
the grade progress he had made prior to coming to Morse School. Iriv:!- 
fact, he progressed at a more rapid rate during the second year of 
his stay at Morse than he did during his first year. As stated above 
in the Results section, Morse School will continue to shortchange 
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if self if it does not analyze the data sufficiently to identify the 
good work it is doing. 

B. Discussion of Role of Evaluators 

As stated in Section A above, one of the disadvantages of Project 
Advance's internal evaluation design was that it did not allow for the 
determination of causal relationships between specific program activi^ 
ties (or even the program itself) and student progress. It was deter- 
mined by the evaluators that (a) Moi^se School implemented the majority 
of the program components described in the Project Advance proposal, 
(b) Morse students participated in these activities, and (c) the stu- 
dents seemed to have improved both academically and in terms of social 
adjustment. However, it is not known whether (o) is a result of (a) 
and (b) - that is, that participation in certain program-activities- 
resulted in improvement in, for example, academic effort or achieve- 
ment . 

What is needed in order to make such a determination is the use 
of control and comparison groups. That is, some students would parti--^ 
cipate in Project Advance activities and others, also identified as 
unable to function in the regular school system, would not (they might 
perhaps be placed on a waiting list control to be admitted to the 
activities later) . Comparisons of the progress of these two groups 
of students after a specific time period could then be made. The 
students who did participate in the program might have changed in ways 
(hopefully, positive ways) that the students who did not attend Morse 
did not. This would then tell us that participation in the Project 
Advance program results in changes in specific behaviors in students. 
With enough eligible students different patterns of activities could 
be offered to several groups of students. 

Such an evaluation design could be taken a step further. A 
design could be developed that would let us determine which, if any, 
of the different components of the program (i.e. the Academic, the 
Psychotherapeutic, etc») are effecting students in specific ways. 
For example :j let us start with a group of 25 students who have been 
identified as unable to fvinction in the regular public school system* 
Let us randomly assign them in groups of five students. Let us leave 
five students perhaps on a waiting list in the regular school system. 
They will receive no additional treatment. Let us leave an additional 
five students in the regular school system but provide them with 
psychotherapeutic counseling at that school site that is identical to 
what they would receive at Morse School. The third group of five 
students would go to Morse School and participate only in its academic 
program. The fourth group would go to Morse School and participate 
in the psychotherapeutic counseling program. The fifth group would 
go to Morse School and would participate in both the psychotherapeutic 
counseling program £Uid the academic program. By comparing through 
use of assessment procedures already in use at Morse, the differences 
and similarities among these five groups, we could determine whether 
a specific program component (or set of components) was beneficial to 
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the students. 

Such an evaluation design, although not complex, requires much 
forethought, planning, and coox^ination among schools and school 
officials. A plan for the development and implementation of such an 
evaluation design must be constructed early in the life of the pro- 
gram. It should, in fact, be developed well before the program begins. 

This is an area where a coordinated effort on the part of the 
inside and outside evaluation teams can be most beneficial. The in- 
side evaluator has the opportunity to do an in«^epth and sensitive 
study because of his closeness to and familiarily with the project. 
An even mo3?e important reason for the special role of the inside 
evaluator is that, being part of the team that designs and operates 
the project i he can be sure that the design and operation of- the^ 
project permits a meaningful evaluation. For example, care must be 
taken (1) to define comparison and control groups, (2) to obtain 
crucial measures before the start of the program, and (3) as appro- 
priate, to make provision for these measures at or after the end of 
the program. 

There is a need for the services of the outside evaluation con- 
sultant, who can serve in an advisory capacity during the initial 
planning and development of the program and can assist the inside 
evaluator in determining whether program guidelines are being met 
and objectives achieved. The emphasis, however, must be on the 
development of a working relationship early in the life of the pro- 
gram so that the necessary coordination between the inside and out- 
side evaluation components occurs. 

Recommendations 

1. Continue with existing academic program. Begin to develop learning 
stations and an integrated educational technology appropriate to 
an Open ClassiToom approach. 

2c Incorporate behaviorally oriented behavior management and teaching 
techniques into the repertoires of the Morse School staff. This 
will involve providing them with in-service training in the theory 
and use of behavior management techniques. 

3. Continue with the use of contingency-based program activities for 
students . Develop academic and social skills programs Idiat are 
built on a series of sequential steps and that include cles^ 
defined goals and appropriate reinforcers for success for all 
students. 

4. Continue with existing psychotherapeutic program. Offer additional 
training opportunities (i.e. one-to-one supervision, seminars) 

to Guided Group Interaction leaders to maintain arid improve their 
leadership and group counseling skills . 
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5. Continue with procedures used to assess a student's readiness 
for return to a regular school. Develop standardized criteria 
for defining "improvement". 

6. Continue vrith existing program of enrichment activities. Place 
additional emphasis on career development opportunities when 
selecting sites for field trips. Allow the students to have 
some input into the selection of field trip sites and activities. 

7. Continue with existing parent and communiiy involvement program. 

8. Continue to develop lines of communication to schools where Morse 
students are placed when they are ready to return to a regular 
program. Try to increase Morse staff input into the decision- 
making concerning student placements, 

9. Continue with follow-up activities, particularly direct contacts 
with the students themselves. 

10. Upgrade the evaluation of the program. Develop an evaluation 
design that defines and uses control and comparison groups. In 
order to develop such an evaluation design, a working relation- 
ship between the inside and outside evaluation teams must be 
established very early in the life of the program. This is nec- 
essary to insure that coordination between inside and outside 
evaluation components occurs and information needed to malce 
decisions about the utility of the program and its components 
is obtained. 

IV. Summary 

An evaluation of the "Project Advance" program at the Morse Crisis 
Intervention Center was conducted. Review of documents, formal and in- 
formal interviews, questionnaires, and direct observations were the methods 
developed and used to assess the correspondence between the objectives of 
the program and Its accomplishments. 

Five students and six members of the teaching and administrative fac- 
ulty at Morse School were interviewed. Various other school personnel were 
interviewed informally. Telephone Interviews were conducted with the par- 
ents of four Morse students. Interviews were conducted with several persons 
who worked in commercial business establishments in the Morse School neigh- 
borhood. A paper and pencil questionnaire was administered to the majority 
of the students. A similar paper and pencil questionnaire was administered 
to the school personnel. Direct observations of classroom and hall activi- 
ties, guided group interaction sessions, participant Interactions, materials, 
equipment, and the physical facility were made. 

The findings derived from formal analyses of available data showed a 
program that achieved a high proportion of what it set out to achieve. An 
area that needs additional attention is the internal assessment of students' 
progress and of the functioning of the program as a whole. A discussion 
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^ of ways of improving the evaluation of this program and programs in general, 
with emphasis on the role of the outside evaluator as consultant to the 
inside evaluation team, is included in this report. The results of the 
analyses of the data^ conclusions, and recommendations to contii.ue some of 
the practices while modifying others are provided in this final evaluation 
report. 
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